Comprehensive Evaluation of Large Language Models for Automated Commit Message Generation: Insights and Improvements

Large Language Models demonstrate superior performance compared to state-of-the-art commit message generation approaches, with GPT-3.5 leading among LLMs. However, LLMs tend to convey more about "What" information but less detailed concerning "Why" information due to limited code context.