This work introduces the concept of theme interpretation as a task in natural language processing (NLP), framed within the context of inferential/interpretive reading comprehension. The authors present the EduStory dataset, the first dataset specifically designed for interpretive comprehension of educational narratives, providing corresponding well-edited theme texts.
The dataset covers a variety of genres and cultural origins, including fables, folk stories, idiom stories, and miscellaneous educational narratives. The stories are annotated with theme keywords based on a taxonomy of character virtues and strengths from positive psychology.
The authors formulate several NLP tasks under different abstractions of interpretive comprehension, including theme keyword identification, story-theme matching, and story reading comprehension on themes. Extensive experiments are conducted using state-of-the-art machine learning and language models, revealing that interpreting themes from narrative text remains a challenging task.
Additionally, the authors explore theme generation using large language models and conduct human evaluation to assess the quality of the generated themes. The results show that while current language models demonstrate strong capabilities, they still fall short of human-level performance in providing concise and accurate theme interpretations.
Overall, this work serves as an initial call to the NLP community to further explore and reflect on reading comprehension issues beyond literal understanding. The EduStory dataset and the proposed tasks provide a valuable resource for advancing research in this direction.
إلى لغة أخرى
من محتوى المصدر
arxiv.org
استفسارات أعمق