Evaluating Zero-shot Cross-lingual Transfer in Instruction Tuning of Large Language Models
Cross-lingual transfer can happen successfully in Instruction Tuning even if all stages of model training are English-centric, but only if multilinguality is taken into account in hyperparameter tuning and with large enough Instruction Tuning data.
