Centrala begrepp
The adoption of GitHub Copilot, an AI-powered code generation tool, significantly improves software engineering productivity and code quality at ANZ Bank, though its impact on code security remains inconclusive.
Sammanfattning
This study explores the integration of the AI tool GitHub Copilot within the software engineering practices at ANZ Bank, a large organization with over 5,000 engineers. The key findings are:
Productivity: The group using GitHub Copilot completed their tasks 42.36% faster on average compared to the control group. This productivity improvement was observed across all skill levels, with the greatest benefit for beginner and intermediate Python programmers.
Code Quality: The code produced by the Copilot group had fewer bugs and code smells, indicating improved maintainability and reliability. However, the impact on code security was inconclusive due to the limited scope of the security-related tasks.
Sentiment: Participants had an overall positive sentiment towards GitHub Copilot, reporting that it had a "positive effect" on their ability to review, test, and document code. They felt the suggestions provided were "somewhat helpful" and aligned well with their coding standards.
The study was conducted over a 6-week period, with the first 2 weeks dedicated to preparation and the remaining 4 weeks for active experimentation. During the experiment, participants were divided into control and Copilot groups to statistically analyze the tool's impact. The data was collected from various sources, including GitHub Copilot metrics, surveys, static code analysis, and grading of code correctness.
While the sample size was limited, the findings suggest that the adoption of GitHub Copilot can significantly enhance software engineering productivity and code quality within a large corporate environment like ANZ Bank. The study provides valuable insights for organizations considering the integration of AI-powered tools in their software development processes.
Statistik
The Copilot group completed tasks 42.36% faster on average compared to the control group.
The Copilot group had a 12.86% higher unit test success ratio compared to the control group, though this result was not statistically significant.
The code produced by the Copilot group had significantly fewer bugs and code smells compared to the control group.
Citat
"GitHub Copilot functions as an advanced assistant for software developers, powered by artificial intelligence (AI). It is adept at generating syntactically correct and contextually relevant code snippets across a diverse array of programming languages."
"Results showed a notable boost in productivity and code quality with GitHub Copilot, though its impact on code security remained inconclusive."
"Participant responses were overall positive, confirming GitHub Copilot's effectiveness in large-scale software engineering environments."