Główne pojęcia
TOPRO improves zero-shot cross-lingual transfer in token-level sequence labeling tasks by utilizing prompt-based learning.
Streszczenie
The article introduces TOPRO, a method that decomposes input sentences into tokens and applies prompts to each token for sequence labeling tasks. It outperforms Vanilla and Prompt-Tuning in zero-shot cross-lingual transfer, especially for languages different from English. TOPRO shows potential as a benchmarking method for evaluating multilingual large language models in sequence labeling tasks.
Statystyki
Our experiments show that TOPRO-based fine-tuning outperforms Vanilla Fine-Tuning and Prompt-Tuning by 19.18% and 25.16% on PAN-X with mBERT.
On UDPOS, TOPRO outperforms Vanilla Fine-Tuning and Prompt-Tuning by 5.27% and 6.24% with mBERT.
The performance improvement of TOPRO is generally more obvious in the cross-lingual context, especially for languages that are linguistically very different from English.
Cytaty
"Prompt-based methods reformulate downstream tasks as language modeling tasks using prompts comprising a template and a set of label words."
"Our experiments show that TOPRO outperforms the baselines with the MPLMs and achieves SOTA performance with mT5."