Core Concepts
This paper introduces EthioMT, a new parallel corpus for 15 low-resource Ethiopian languages paired with English, and presents benchmark results for machine translation performance on this corpus using transformer and fine-tuning approaches.
Abstract
The paper discusses the creation of the EthioMT parallel corpus, which covers 15 Ethiopian languages from the Afro-Asiatic and Nilo-Saharan language families. It provides details on the languages included, their language families, number of speakers, and dataset sizes.
The authors collected datasets for the languages, primarily from religious domains, and aligned the sentences with their English translations. They then preprocessed the data and split it into training, development, and test sets.
To evaluate the usefulness of the new corpus, the authors conducted baseline experiments using two approaches: a transformer model and fine-tuning a multilingual M2M100-48 model. The results show that the fine-tuning approach outperformed the transformer model in both translation directions (from English to Ethiopian languages and vice versa). The performance was better for languages with larger dataset sizes, such as Amharic, Afaan Oromo, Somali, and Tigrinya, compared to languages with smaller datasets.
The authors conclude that the EthioMT corpus can foster collaboration and facilitate research and development in low-resource Ethiopian languages. They plan to expand the corpus size and explore additional machine translation approaches in the future.
Stats
Amharic has a dataset size of 1.5M.
Afaan Oromo has a dataset size of 2.9M.
Somali has a dataset size of 1.2M.
Tigrinya has a dataset size of 140K.
Quotes
"Recent research in natural language processing (NLP) has achieved impressive performance in tasks such as machine translation (MT), news classification, and question-answering in high-resource languages. However, the performance of MT leaves much to be desired for low-resource languages."
"Ethiopia is a country that stands out for its remarkable cultural and linguistic diversity, with over 85 spoken languages. Only a few languages of Ethiopia have received attention in the area of NLP research and application development. Most languages have been left behind due to resource limitation."