المفاهيم الأساسية
This paper introduces MathCoder2, a family of large language models (LLMs) with enhanced mathematical reasoning abilities achieved through a novel continued pretraining method using model-translated mathematical code paired with natural language reasoning steps.
الإحصائيات
MathCode-Pile consists of 19.2B tokens.
The model-translated mathematical code constitutes 14.1% of MathCode-Pile.
MathCoder2-Llama-3-8B achieved 4-shot accuracies of 38.4% on MATH and 69.9% on GSM8K.
The baseline model achieved 4-shot accuracies of 35.3% on MATH and 65.8% on GSM8K.
اقتباسات
"Reasoning with the help of code is particularly effective for more challenging problems, likely due to its precision and accuracy."
"When used in pretraining, the mathematical code paired with reasoning steps facilitates LLMs’ understanding of math-related pretraining texts, as it effectively captures the underlying reasoning process."
"This openness facilitates transparency, reproducibility, and further research, which is crucial for advancing the field."