Comprehensive Analysis of Large Language Model Pretraining and Downstream Capabilities

This paper undertakes a comprehensive analysis of the dynamic changes in capabilities of large language models during the pretraining process. It identifies patterns in the learning trajectories of various downstream tasks and provides insights to guide the optimization of pretraining strategies.