ContrastRepair: Enhancing Automated Program Repair with Contrastive Test Cases

核心概念

ContrastRepair significantly improves program repair efficiency by providing informative prompts to Large Language Models through contrastive test cases.

摘要

ContrastRepair introduces a novel approach to automated program repair by leveraging Large Language Models (LLMs) and contrastive test pairs. The method aims to enhance the effectiveness of LLMs in accurately localizing bugs and generating high-quality fixes. By providing both negative and positive feedback, ContrastRepair enables LLMs to pinpoint root causes more effectively. The process involves constructing test pairs consisting of failing and passing tests, selecting suitable passing tests based on similarity metrics, and generating prompts for LLMs. The iterative repair process continues until plausible patches are produced or the repair budget is exhausted. Evaluation on benchmark datasets demonstrates that ContrastRepair outperforms existing methods, achieving new state-of-the-art performance in program repair.

统计

Among Defects4j 1.2 and 2.0, ContrastRepair correctly repairs 143 out of all 337 bug cases.
ContrastRepair successfully resolved 360 out of 581 bugs in multiple benchmark datasets.
An average reduction of 20.91% in API calls compared to CHATREPAIR was observed.

引用

"By providing comparative input pairs that juxtapose the outcomes of these tests, LLMs are more likely to effectively localize the root cause."
"Our key insight lies in that relying solely on negative feedback may not always be adequate for LLMs to precisely pinpoint the bugs."
"We propose a method that aims to craft more specific and informative prompts for enhancing the capabilities of LLMs in accurately localizing bugs."

从中提取的关键见解

ContrastRepair

by Jiaolong Kon... 在 arxiv.org 03-05-2024

https://arxiv.org/pdf/2403.01971.pdf

更深入的查询

質問1

ContrastRepairのアプローチは、自動プログラム修復以外の他の領域にどのように適用できますか？
Answer 1 here

質問2

ChatGPTなどの大規模言語モデルに重点を置くことが、プログラム修復において潜在的な制限や欠点を引き起こす可能性がありますか？
Answer 2 here

質問3

対照的フィードバックのコンセプトは、他の機械学習タスクやアプリケーションでどのように活用される可能性がありますか？
Answer 3 here