Measuring and Understanding the Impact of Evaluation Data Contamination in Large Language Models

Core Concepts

Evaluation data contamination, though difficult to precisely define and measure, can significantly inflate benchmark scores of large language models (LLMs), and the impact varies with model scale and benchmark type.

Abstract

• Bibliographic Information: Singh, A. K., Kocyigit, M. Y., Poulton, A., Esiobu, D., Lomeli, M., Szilvasy, G., & Hupkes, D. (2024). Evaluation data contamination in LLMs: how do we measure it and (when) does it matter? arXiv preprint arXiv:2411.03923v1.
• Research Objective: This paper investigates the impact of evaluation data contamination on large language model (LLM) benchmark scores and proposes a novel method, ConTAM, to assess contamination metrics based on their impact on model performance.
• Methodology: The authors analyze four n-gram based contamination metrics (NGRAM-MATCH, TOKEN-MATCH, TOKEN-EXTEND, and LONGEST-MATCH) across 13 benchmarks and 7 LLMs of varying sizes trained on two different pre-training corpora. They introduce the concept of Estimated Performance Gain (EPG) to quantify the impact of contamination on benchmark scores and use z-scores to select optimal contamination thresholds for each model-benchmark pair.
• Key Findings: The study reveals that evaluation data contamination is more prevalent and impactful than previously reported, especially for larger LLMs. The LONGEST-MATCH metric, which considers only the longest contaminated substring, proves to be more effective in detecting meaningful contamination across various benchmarks. The analysis also highlights the importance of model-specific threshold selection and the influence of hyperparameters like n-gram size (n) and minimum frequency (mincount) on contamination detection.
• Main Conclusions: The authors argue for a more nuanced understanding of evaluation data contamination and its impact on LLM evaluation. They emphasize the need for careful selection of contamination metrics and hyperparameters, advocating for empirical grounding of these choices in downstream performance effects.
• Significance: This research provides valuable insights into the challenges of evaluating LLMs in the presence of data contamination and offers practical recommendations for researchers and practitioners to mitigate its impact.
• Limitations and Future Research: The study primarily focuses on n-gram based contamination metrics and two specific pre-training corpora. Future research could explore alternative contamination detection methods and analyze a wider range of pre-training datasets and LLM architectures.

Stats

For 8 out of 13 benchmark datasets, on average, over 50% of the samples are marked as contaminated in the Llama 1 pre-training corpus.
The largest Llama model exhibits an estimated performance gain (EPG) of over 15% on both HumanEval and Big Bench Hard benchmarks due to contamination.
Three additional datasets (HellaSwag, MMLU, and PiQA) show an EPG of 10 points or higher for the largest Llama model.

Quotes

"Evaluation data contamination, the inadvertent mixing of samples from evaluation benchmarks into pre-training corpora, constitutes a recently growing and important concern in the field of evaluating large language models (LLMs)."
"The impact of evaluation data contamination has been underestimated in many prominent LLM releases, likely because of false negatives in the chosen contamination metrics."
"While there is no true one-size-fits-all approach to contamination detection, using the longest contaminated substring rather than a union of all matches works better across the board, adequately detecting contamination in cases where no other metric did."

Key Insights Distilled From

Evaluation data contamination in LLMs: how do we measure it and (when) does it matter?

by Aaditya K. S... at arxiv.org 11-07-2024

https://arxiv.org/pdf/2411.03923.pdf