Quantifying Data Contamination in Code Generation Benchmarks

The author explores the impact of data contamination on code generation benchmarks, highlighting the overlap between training data and evaluation benchmarks. By quantifying this overlap, the study sheds light on how models perform better when exposed to similar solutions during training.