Massively Parallel Cross-Lingual Learning in Low-Resource Target Language Translation

Abstract

We work on translation from rich-resource languages to low-resource languages. The main challenges we identify are the lack of lowresource language data, effective methods for cross-lingual transfer, and the variable-binding problem that is common in neural systems. We build a translation system that addresses these challenges using eight European language families as our test ground. Firstly, we add the source and the target family labels and study intra-family and inter-family influences for effective cross-lingual transfer. We achieve an improvement of +9.9 in BLEU score for English-Swedish translation using eight families compared to the single-family multi-source multi-target baseline. Moreover, we find that training on two neighboring families closest to the low-resource language is often enough. Secondly, we construct an ablation study and find that reasonably good results can be achieved even with considerably less target data. Thirdly, we address the variable-binding problem by building an order-preserving named entity translation model. We obtain 60.6% accuracy in qualitative evaluation where our translations are akin to human translations in a preliminary study.

APE through Neural and Statistical MT with Augmented Data. ADAPT/DCU Submission to the WMT 2019 APE Shared Task

“…In their case, the source-language token is used for language specific tokenisation. Similarly, Zhou et al (2018) found that adding tokens that encode the source and target language family, e. g. source-family:Germanic and target-family:Slavic for English-Czech translation, may improve the accuracy of the NMT outputs for low-resource languages.…”

Paraphrases as Foreign Languages in Multilingual Neural Machine Translation

“…The two multilingual baselines are strong baselines while the fWMT baseline is a weak baseline that helps us to evaluate all experiments' performance discounting the effect of increasing data. All baseline results are taken from a research work which uses the grid of (1,6,11,16,22) for the number of languages or equivalent number of unique sentences and we follow the same in Figure 3 (Zhou et al, 2018). All experiments for each grid point carry the same number of unique sentences.…”

“…Moreover, we have two multilingual baselines 2 built on multilingual attentional NMT, Family and Span (Zhou et al, 2018). Family refers to the multilingual baseline by adding one language family at a time, where on top of the French corpus f0 and the English corpus e0, we add up to 20 other European languages.…”

“…Implicit parameter sharing enables multilingual NMT to learn across languages and achieve better generalization (Johnson et al, 2017). Training on closely related languages are shown to improve translation (Zhou et al, 2018). We view paraphrases as an extreme case of closely related languages and view multilingual data as paraphrases in different languages.…”

The University of Maryland’s Kazakh-English Neural Machine Translation System at WMT19

“…In this setting, an NMT system is firstly trained using auxiliary parallel data from a so-called "parent" language pair and then the trained model is used to initialize a "child" model which is further trained on a low-resource language pair. Similar approaches that support cross-lingual transfer learning for Multi-lingual NMT train a model on the concatenation of all data instead of employing sequential training (Gu et al, 2018;Zhou et al, 2018;Wang et al, 2019).…”