แนวคิดหลัก
Large Language Models may generate off-topic answers in Open Domain Multi-Hop Question Answering, impacting performance.
บทคัดย่อ
The article introduces the Dr3 mechanism to address off-topic answers in ODMHQA. It highlights the importance of accurate answers and the challenges posed by off-topic responses. The proposed solution involves a Discriminator and Corrector to detect and correct off-topic answers, improving performance significantly.
Abstract:
Large Language Models (LLMs) excel at Open Domain Multi-Hop Question Answering (ODMHQA).
However, LLMs may generate off-topic answers, affecting accuracy.
The Dr3 mechanism aims to reduce off-topic answers through post-hoc judgment and corrections.
Introduction:
ODMHQA requires multi-step reasoning over external knowledge sources.
LLMs like ReAct prompt complex problem-solving but face issues with off-topic answers.
Method:
Dr3 consists of a Discriminator to judge on-topicness and a Corrector for step-wise revisions.
Experimental results show Dr3 reduces off-topic answers by nearly 13%.
Results:
Dr3 outperforms baselines on HotpotQA and 2WikiMultiHopQA datasets.
The Discriminator achieves high accuracy in detecting off-topic answers.
Related Work:
Previous research focused on enhancing reasoning capabilities in LLMs for QA tasks.
Post-hoc correction methods have been explored for improving text generation quality.
สถิติ
Approximately one-third of incorrect answers are identified as off-topic.
Experimental results show Dr3 reduces occurrence of off-topic answers by nearly 13%.
Off-topic ratio increases with the number of Sub-Questions in reasoning chains.