Sung, M., Feng, S., Gung, J., Shu, R., Zhang, Y., & Mansour, S. (2024). Structured List-Grounded Question Answering. arXiv preprint arXiv:2410.03950.
This research aims to address the limitations of current document-grounded dialogue systems in effectively handling structured list data for question answering. The authors introduce a new dataset and method to improve the ability of QA systems to understand and leverage list information.
The authors developed LIST2QA, a dataset created from customer service documents containing various list types (conditions, steps, options, non-action information). They employed large language models (LLMs) for automated data creation, simulating user queries and system responses grounded in list information. Additionally, they propose the Intermediate Steps for Lists (ISL) method, which explicitly models structured list data and user contexts to enhance response generation. The researchers fine-tuned smaller LLMs (Flan-T5-XL and Mistral-7B-Instruct) on LIST2QA and compared their performance against larger LLMs (GPT-3.5 and Mixtral-8x7B-Instruct) using metrics like ROUGE-L, correctness, faithfulness, and completeness.
This research highlights the importance of explicitly modeling structured list data and user contexts for improving question answering systems. The proposed LIST2QA dataset and ISL method provide valuable resources for advancing research in list-grounded question answering.
This work significantly contributes to the field of natural language processing by addressing the under-explored area of list-grounded question answering. The proposed dataset and method can be valuable resources for developing more sophisticated and robust QA systems.
The study acknowledges limitations in handling diverse logical relations beyond "and" and "or" in conditional lists and focuses on single-turn QA tasks. Future research could explore more complex logical relations, multi-turn dialogues, and develop more cost-effective and accurate evaluation methods for list-grounded question answering systems.
To Another Language
from source content
arxiv.org
Key Insights Distilled From
by Mujeen Sung,... at arxiv.org 10-08-2024
https://arxiv.org/pdf/2410.03950.pdfDeeper Inquiries