The authors conduct an extensive study on the effectiveness of Low-Rank Adaptation (LoRA) for multilingual summarization across different data availability scenarios, including high-data, low-data, and cross-lingual transfer settings.
In the high-data regime, the authors find that full fine-tuning achieves the best ROUGE-L scores, but LoRA exhibits superior performance in terms of summary faithfulness and conciseness. As the amount of training data decreases, LoRA becomes a better alternative to full fine-tuning, delivering competitive or even superior results while being more computationally efficient.
In the cross-lingual transfer setting, the authors explore zero-shot and few-shot learning. They observe that LoRA consistently outperforms full fine-tuning, especially when only a small number of target language examples are available. The authors also investigate different strategies for few-shot cross-lingual transfer, finding that continued LoRA tuning outperforms full fine-tuning and the dynamic composition of language-specific LoRA modules.
When scaling up to the larger PaLM 2-S model, LoRA achieves on-par performance with full fine-tuning, making it a better choice due to its computational efficiency.
다른 언어로
소스 콘텐츠 기반
arxiv.org
더 깊은 질문