
Ziya2, a 13-billion-parameter language model, is developed through a data-centric approach that focuses on optimizing the use of pre-training data to enhance the model's capabilities in Chinese, mathematics, and programming tasks, while maintaining or improving its performance on general English benchmarks.


coremsg

ziya2-a-data-centric-approach-to-enhance-large-language-models-capabilities-in-chinese-mathematics-and-programming


Ziya2: A Data-Centric Approach to Enhance Large Language Models' Capabilities in Chinese, Mathematics, and Programming


title_rewrite


The LLM-ADE framework introduces a novel approach to continual pre-training of large language models, enabling efficient integration of new datasets while mitigating catastrophic forgetting and double descent.


enhancing-large-language-models-through-adaptive-data-engineering-the-llm-ade-framework


Enhancing Large Language Models through Adaptive Data Engineering: The LLM-ADE Framework
