CodeBenchGen: A Scalable Framework for Creating Execution-based Code Generation Benchmarks

CodeBenchGen is a framework that leverages large language models to automatically convert arbitrary code fragments into executable evaluation examples, enabling the creation of custom and scalable code generation benchmarks.