Temel Kavramlar
MAGIS proposes a novel LLM-based Multi-Agent framework for GitHub Issue resolution, significantly outperforming popular LLMs.
Özet
The content introduces MAGIS, a framework for resolving GitHub issues using Large Language Models (LLMs). It addresses challenges faced by LLMs in code change tasks at the repository level. The framework consists of four agents: Manager, Repository Custodian, Developer, and Quality Assurance Engineer. Experiments show MAGIS outperforms GPT-4, achieving a resolved ratio of 13.94%. Factors like line location and code complexity impact issue resolution rates.
Abstract
LLMs face challenges in resolving GitHub issues at the repository level.
MAGIS proposes a Multi-Agent framework leveraging LLMs for issue resolution.
Experiments show MAGIS significantly outperforms popular LLMs.
Introduction
Software evolution requires addressing emergent bugs and adapting to new requirements.
GitHub issues signify the need for software evolution.
LLMs excel in code generation but face challenges in advanced tasks like GitHub issue resolution.
Methodology
MAGIS framework involves four agents collaborating in planning and coding.
Planning phase involves locating code files and team building.
Coding phase includes developers generating code changes and QA engineers reviewing them.
Experiments and Analysis
MAGIS significantly outperforms GPT-4 and Claude-2 in resolving GitHub issues.
Planning process effectiveness demonstrated through repository custodian and project manager agent analysis.
Coding process effectiveness analyzed through line location overlap and complexity indices correlation.
İstatistikler
MAGIS can resolve 13.94% of GitHub issues, outperforming GPT-4 significantly.
Alıntılar
"LLMs excel in generating function-level code but face challenges in code change tasks."
"MAGIS achieves an eight-fold increase in resolved ratio over GPT-4."