WikiFactDiff: A Large Dataset for Factual Knowledge Update
Grunnleggende konsepter
Large language models require factual knowledge updates to stay relevant and accurate.
Sammendrag
Introduction to the need for factual knowledge updates in large language models.
Description of WikiFactDiff dataset creation process, including preprocessing, classification rules, and neighbor fact selection.
Evaluation of existing update algorithms on the WFDrepl subset of WikiFactDiff.
Comparison with CounterFact dataset results and discussion on bleedover detection methods.
Conclusion highlighting the importance of realistic update scenarios and future research directions.
WikiFactDiff
Statistikk
"The factuality of large language model (LLMs) tends to decay over time since events posterior to their training are “unknown” to them."
"WikiFactDiff constitutes a realistic update setting that involves various update scenarios, including replacements, archival, and new entity insertions."
"The release of WikiFactDiff spans the evolution of factual knowledge between 4 January 2021 and 27 February 2023."