Enhancing Transformer In-Context Learning Capabilities through Multi-Task Training and Curriculum Learning Strategies

Curriculum learning strategies, particularly a mixed curriculum approach, can improve the data efficiency and convergence of transformer models in learning multiple function classes through in-context learning.