Core Concepts
Language models show promising ability to utilize auxiliary functions, but improvements are needed for better implementation.
Abstract
The study explores the utilization of auxiliary functions in language models for code generation. It introduces the HumanExtension dataset, evaluates the effectiveness and robustness of including auxiliary functions in prompts, and analyzes implementation styles. Results show varying abilities of models to utilize auxiliary functions, with a preference for black-box style implementations. The study highlights the need for further research to enhance model capabilities in utilizing auxiliary functions effectively.
Stats
"We collect 151 problems representing a function pair that one function extends the other and name it HumanExtension."
"Our experimental results show current LLMs’ capabilities to utilize auxiliary function and their limitations."
Quotes
"We release our code1 and dataset2 to facilitate this research direction."
"Models exhibit large performance improvement with proper relevant auxiliary functions."