The Helsinki submission to the AmericasNLP shared task

Abstract

The University of Helsinki participated in the AmericasNLP shared task for all ten language pairs. Our multilingual NMT models reached the first rank on all language pairs in track 1, and first rank on nine out of ten language pairs in track 2. We focused our efforts on three aspects: (1) the collection of additional data from various sources such as Bibles and political constitutions, (2) the cleaning and filtering of training data with the OpusFilter toolkit, and (3) different multilingual training techniques enabled by the latest version of the OpenNMT-py toolkit to make the most efficient use of the scarce data. This paper describes our efforts in detail.

Improving Neural Machine Translation of Indigenous Languages with Multilingual Transfer Learning

“…BLEU and chrF are the two metrics adopted by AmericasNLP 2021 Shared Task. We surpassed the winner of Americas-NLP2021 (Vázquez et al, 2021), in either or both metrics, for 5 language pairs with the following languages as target: Bribri (bzd), Asháninka (cni), Wixarika (hch), Nahuatl (nah), and Hñähñu (oto). Notably, we double the performance in BLEU score for es-bzd, increasing by about 7.7 BLEU scores and 0.1 chrF.…”

“…We are able to surpass previous SOTA in five language pairs and mBART50 curr achieves 7.143 BLEU and 0.3134 chrF on average, comparing to previous SOTA having 6.693 BLEU and 0.3171 chrF on average. It can be hypothesized that the reason why we are able to improve average BLEU score by 0.45, accomplish comparable average chrF, and surpass in five language pairs is because we use an MT model pretrained on 50 languages, while Vázquez et al (2021) pretrain their model only on es-en. We suspect that there could be some languages, other than Spanish and English, which contribute to positive transfer to Indigenous languages.…”

“…Nagoudi et al (2021) create models based on the T5 architecture (Raffel et al, 2019) and train it with monolingual Indigenous data before finetuning on parallel data, thus attempting to acquire knowledge of the Indigenous languages to benefit MT. Vázquez et al (2021) augment the parallel data by collecting monolingual Indigenous data and translate them to Spanish. They then train a multilingual MT model with the augmented parallel data.…”

“…We suspect that there could be some languages, other than Spanish and English, which contribute to positive transfer to Indigenous languages. Unlike Vázquez et al (2021), we do not leverage external data to build a larger train set. Nor do we build a single multilingual model for all 10 language pairs, but we rather train one model for each language pair (where every single Pair Sentence es-aym nanakan utaxax khaysa Concord uksanx kimsatunka waranqa acres ukhamarac walja uywanakarakiw utjaraki.…”

“…Our downstream datasets are provided by Americas-NLP2021 (Mager et al, 2021) shared task. We compare our performance to the winner of the shared task (Vázquez et al, 2021).…”