On Romanization for Model Transfer Between Scripts in Neural Machine Translation

Abstract

Transfer learning is a popular strategy to improve the quality of low-resource machine translation. For an optimal transfer of the embedding layer, the child and parent model should share a substantial part of the vocabulary. This is not the case when transferring to languages with a different script. We explore the benefit of romanization in this scenario. Our results show that romanization entails information loss and is thus not always superior to simpler vocabulary transfer methods, but can improve the transfer between related languages with different scripts. We compare two romanization tools and find that they exhibit different degrees of information loss, which affects translation quality. Finally, we extend romanization to the target side, showing that this can be a successful strategy when coupled with a simple deromanization model.

When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Models

“…6 Our findings are generally in line with previous work. Transliteration to English specifically (Lin et al, 2016;Durrani et al, 2014) and named entity transliteration (Kundu et al, 2018;Grundkiewicz and Heafield, 2018) has been proven useful for cross-lingual transfer in tasks like NER, entity linking (Rijhwani et al, 2019), morphological inflection (Murikinati et al, 2020), and Machine Translation (Amrhein and Sennrich, 2020).…”

Arabic Machine Translation: A Survey With Challenges and Future Directions

“…Additionally, when comparing our findings and anticipated AMT results to other languages pairs such as Czech-English and vice-versa, we observe followings:  Although both languages; Arabic and Czech are morphologically-rich [68], have free word order, and share most of the technical challenges, we observe with regard to the latest WMT20 findings [128], a constant improvement in the baseline systems for Czech MT in contrast to AMT.  Romanized-based models [106] , [129] reduce vocabulary sizes and UNK rates and achieve better or comparable translation results compared with their counterparts in Arabic under various segmentation scenarios. While the advantage of romanization at the subword level is that Latin encoding provides great flexibility in extracting proper BPE rules during segmentation, further reducing rare words and improving translation quality.…”

Finite-state script normalization and processing utilities: The Nisaba Brahmic library

“…The main fo cus of these libraries is script conversion and ro manization. In this capacity they were success fully employed in diverse downstream multilin gual NLP tasks such as neural machine transla tion (Zhang et al, 2020; Amrhein andSennrich, 2020), morphological analysis (Hauer et al, 2019; Murikinati et al, 2020, named entity recogni tion (Huang et al, 2019) and partofspeech tag ging (Cardenas et al, 2019).…”