EasyProject: A Robust and Effective Mark-then-Translate Approach for Cross-lingual Transfer Learning
A novel mark-then-translate method, EasyProject, simplifies cross-lingual transfer learning for span-level NLP tasks by outperforming traditional word alignment methods while being easier to implement.
Improving Cross-Lingual Transfer in Decoder Language Models Through Pretraining with Active Forgetting
Pretraining decoder-only LLMs with active forgetting, a technique involving periodic resetting of token embeddings, enhances their cross-lingual transfer capabilities and allows for better adaptation to new languages without sacrificing performance in other languages.
Inference-Time Cross-Lingual Intervention for Improved Language Model Performance in Low-Resource Languages
INCLINE, a novel inference-time intervention framework, effectively bridges performance gaps between high-resource and low-resource languages in Large Language Models (LLMs) by aligning their internal representations, leading to significant performance improvements on various multilingual tasks without requiring costly retraining or fine-tuning.
Mitigating the Linguistic Gap with Phonemic Representations for Robust Cross-lingual Transfer: A Comparative Study of Phoneme-Based, Character-Based, and Subword Language Models
Phonemic representations, particularly using the International Phonetic Alphabet (IPA), can mitigate performance gaps in cross-lingual transfer learning by reducing linguistic discrepancies between languages, especially for low-resource languages, as demonstrated by improved results in tasks like XNLI, NER, and POS tagging.
Improving Cross-Lingual Transfer in Multilingual Language Models by Aligning Different Scripts with Transliteration
Transliteration-based post-training alignment improves cross-lingual transfer in multilingual language models, especially between related languages with different scripts, by aligning representations in the original and Latin scripts.
PreAlign: Enhancing Cross-Lingual Transfer in Large Language Models Through Early Multilingual Alignment
Establishing strong multilingual alignment before and during the pretraining of large language models significantly improves their ability to transfer knowledge and skills across languages, especially in low-resource scenarios.
Enhancing Cross-Lingual Transfer of Large Language Models via Self-Translation
Large language models can capture cross-lingual correspondence, which can be effectively elicited through self-translation to improve cross-lingual transfer performance.
Measuring Catastrophic Forgetting in Cross-Lingual Classification Transfer Paradigms and Tuning Strategies
Cross-lingual transfer learning can lead to catastrophic forgetting of previously acquired knowledge in the source language. This study compares different cross-lingual transfer strategies and fine-tuning approaches to measure and mitigate this effect.
Measuring Cross-lingual Knowledge Transfer Across Diverse Languages
The study demonstrates that language models can effectively transfer knowledge across diverse languages, with the transfer being largely independent of language proximity. This suggests the presence of language-agnostic representations that enable cross-lingual generalization.
DA-Net: A Disentangled and Adaptive Network for Multi-Source Cross-Lingual Transfer Learning
다중 소스 크로스-언어 전이 학습을 위한 DA-Net의 효과적인 구조 소개
