Addressing data sparsity for neural machine translation between morphologically rich languages

Abstract

Translating between morphologically rich languages is still challenging for actual machine translation systems. In this paper, we experiment with various Neural Machine Translation (NMT) architectures to address the data sparsity problem caused by data availability (quantity), domain shift and the languages involved (Arabic and French). We showed that the Factored NMT (FNMT) model, which uses linguistically motivated factors, is able to outperform standard NMT systems using subword units by more than 1% BLEU point even when a large quantity of data is available. Our work shows the benefits of applying linguistic factors in NMT when faced with low and large resource conditions.

Challenges in Rendering Arabic Text to English Using Machine Translation: A Systematic Literature Review

“…An attention-based neural machine translation model between Arabic and English is proposed by Almahairi et al [25] that had the highest level of accuracy. More recently, Garcia-Martinez et al [48] examined the results of decomposing the target words of an Arabic-French factored NMT model employing linguistic preprocessing. Their model predicted the lemma as well as the combination of the following elements at the time of decoding: the POS tag, the tense, the gender, the number, the person, and the case information.…”

Arabic Machine Translation: A Survey With Challenges and Future Directions

“…More recently, Martínez et al [102] compared the effect of using linguistic preprocessing to decompose the target words of an Arabic-French factored NMT model. Their model predicted the lemma, and the concatenation of the following factors: POS tag, tense, gender, number, person, and the case information at decoding time.…”

Understanding the effects of word-level linguistic annotations in under-resourced neural machine translation

“…The latter involve producing probability distributions for both TL words and TL linguistic annotations, which can be seen as a form of multi-task learning. Multi-task learning architectures explored in the literature include: independent decoders for words and linguistic annotations (Zhou et al, 2017;Wu et al, 2018;Gū et al, 2018;Wang et al, 2018;Yang et al, 2019); independent output layers in the same decoder (García-Martínez et al, 2016;Grönroos et al, 2017;Feng et al, 2019); and even sharing the same network for both tasks (Nadejde et al, 2017;Tamchyna et al, 2017;Wagner, 2017) and alternatively produce linguistic annotations and words. The latter approach is usually referred to as interleaving.…”

“…We can also classify the approaches according to the type of linguistic annotations used: part-of-speech tags (Feng et al, 2019;Yang et al, 2019); morpho-syntactic description tags, which comprise part of speech and morphological inflection information (García-Martínez et al, 2016;Tamchyna et al, 2017); and syntactic structure information (Nadejde et al, 2017;Zhou et al, 2017;Wu et al, 2018;Gū et al, 2018;Wang et al, 2018). Using morpho-syntactic description tags as TL annotations allows us to train the network to produce TL lemmas instead of surface forms (words as they appear in running texts).…”