核心概念
There is an inherent tension between a large language model's internal prior knowledge and the information presented in retrieved context, which can lead to unpredictable model behavior when the two sources disagree.
摘要
The authors systematically analyze the tug-of-war between a large language model's (LLM) internal knowledge and the retrieved information in a retrieval-augmented generation (RAG) setting. They find that:
- The likelihood of the LLM adhering to the retrieved information (RAG preference rate) is inversely correlated with the model's confidence in its own prior response.
- LLMs are more likely to revert to their priors when the retrieved context is progressively modified with unrealistic values.
These findings hold across 6 different datasets spanning over 1,200 questions, using GPT-4, GPT-3.5, and Mistral-7B models. The authors also show that the choice of prompting technique can influence the strength of this relationship.
The results highlight an underlying tension in LLMs between their pre-trained knowledge and the information presented in retrieved context. This has important implications for the reliability of RAG systems, especially as they are increasingly deployed in high-stakes domains like healthcare and finance. The authors caution that users and developers should be aware of these unintended effects when relying on RAG-enabled LLMs.
統計資料
The model's prior response only agreed with the reference answer 34.7% on average.
Providing the retrieved context elevated the concordance to 94%.
For every 10% increase in the probability of the prior token, there is a 2.3% decreased likelihood of the model preferring the RAG information.
引述
"As the RAG value diverges from the model's prior, the model is less likely to adopt the RAG value over its own initial response."
"The choice of prompt is thus an important mechanism for influencing the LLM's RAG preferences."