Implicit Bias of Next-Token Prediction: Optimization and Generalization Principles in NTP Training Paradigm

NTP training exhibits implicit bias towards specific solutions, impacting optimization and generalization principles.