Centrala begrepp
Retrieval-Augmented Generation (RAG) models are susceptible to hallucinations stemming from inaccurate retrieval results. This paper introduces Corrective Retrieval Augmented Generation (CRAG), a novel method to enhance the robustness of RAG by implementing a self-correction mechanism for retrieved documents and leveraging web searches for knowledge supplementation.
Statistik
CRAG outperformed RAG by margins of 7.0% accuracy on PopQA, 14.9% FactScore on Biography, 36.6% accuracy on PubHealth, and 15.4% accuracy on Arc-Challenge when based on SelfRAG-LLaMA2-7b.
CRAG outperformed RAG by margins of 4.4% accuracy on PopQA, 2.8% FactScore on Biography, and 10.3% on Arc-Challenge when based on LLaMA2-hf-7b.
Compared with Self-RAG, Self-CRAG achieved improvements of 20.0% accuracy on PopQA, 36.9% FactScore on Biography, and 4.0% accuracy on Arc-Challenge when based on LLaMA2-hf-7b.
Compared with Self-RAG, Self-CRAG achieved improvements of 6.9% accuracy on PopQA, 5.0% FactScore on Biography, and 2.4% accuracy on PubHealth, when based on SelfRAG-LLaMA2-7b.
The lightweight T5-based retrieval evaluator outperformed ChatGPT in all settings for assessing retrieval accuracy.
Citat
"LLMs inevitably exhibit hallucinations since the accuracy of generated texts cannot be secured solely by the parametric knowledge they encapsulate."
"While RAG serves as a practicable complement to LLMs, its effectiveness is contingent upon the relevance and accuracy of the retrieved documents."
"This paper particularly studies the scenarios where the retriever returns inaccurate results."
"CRAG is plug-and-play and can be seamlessly coupled with various RAG-based approaches."