Bridging Linguistic Typology and Multilingual Machine Translation with Multi-View Language Representations

Oncevay, Arturo; Haddow, Barry; Birch, Alexandra

doi:10.18653/v1/2020.emnlp-main.187

Cited by 27 publications

(32 citation statements)

References 39 publications

(57 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To construct linguistic distances (Hajič, 2000;Oncevay et al, 2020), some explore typological distance (Chowdhury et al, 2020;Rama and Kolachina, 2012;Pienemann et al, 2005;Svalberg and Chuchu, 1998;Hansen et al, 2012;Comrie, 2005), lexical distance (Huang et al, 2007), Levenshtein distance and Jaccard distance (Serva and Petroni, 2008;Holman et al, 2008;Adebara et al, 2020), sonority distance (Parker, 2012) and spectral distance (Dubossarsky et al, 2020).…”

Section: Linguistic Distancementioning

confidence: 99%

Family of Origin and Family of Choice: Massively Parallel Lexiconized Iterative Pretraining for Severely Low Resource Text-based Translation

Zhou¹,

Waibel²

2021

Proceedings of the Third Workshop on Computational Typology and Multilingual NLP

View full text Add to dashboard Cite

We translate a closed text that is known in advance into a severely low resource language by leveraging massive source parallelism. In other words, given a text in 124 source languages, we translate it into a severely low resource language using only ∼1,000 lines of low resource data without any external help. Firstly, we propose a systematic method to rank and choose source languages that are close to the low resource language. We call the linguistic definition of language family Family of Origin (FAMO), and we call the empirical definition of higher-ranked languages using our metrics Family of Choice (FAMC). Secondly, we build an Iteratively Pretrained Multilingual Orderpreserving Lexiconized Transformer (IPML) to train on ∼1,000 lines (∼3.5%) of low resource data. To translate named entities correctly, we build a massive lexicon table for 2,939 Bible named entities in 124 source languages, and include many that occur once and covers more than 66 severely low resource languages. Moreover, we also build a novel method of combining translations from different source languages into one. Using English as a hypothetical low resource language, we get a +23.9 BLEU increase over a multilingual baseline, and a +10.3 BLEU increase over our asymmetric baseline in the Bible dataset. We get a 42.8 BLEU score for Portuguese-English translation on the medical EMEA dataset. We also have good results for a real severely low resource Mayan language, Eastern Pokomchi.

show abstract

Section: Linguistic Distancementioning

confidence: 99%

Family of Origin and Family of Choice: Massively Parallel Lexiconized Iterative Pretraining for Severely Low Resource Text-based Translation

Zhou¹,

Waibel²

2021

Proceedings of the Third Workshop on Computational Typology and Multilingual NLP

View full text Add to dashboard Cite

show abstract

“…SP08 was constructed by computing the Levenshtein (edit) distance between words from an open cross-lingual list (Dyen et al, 1992) to compare linguistic divergence through time and thus partially encodes lexical similarity of languages (Oncevay et al, 2020). Rabinovich et al (2017) also acknowledges that SP08 has been disputed and researchers have not yet agreed on a commonly accepted tree of the Indo-European languages (Ringe et al, 2002).…”

Section: Phylogenetics and Shining-throughmentioning

confidence: 99%

Understanding Translationese in Multi-view Embedding Spaces

Chowdhury¹,

España-Bonet

Genabith

2020

Proceedings of the 28th International Conference on Computational Linguistics

View full text Add to dashboard Cite

Recent studies use a combination of lexical and syntactic features to show that footprints of the source language remain visible in translations, to the extent that it is possible to predict the original source language from the translation. In this paper, we focus on embedding-based semantic spaces, exploiting departures from isomorphism between spaces built from original target language and translations into this target language to predict relations between languages in an unsupervised way. We use different views of the data -words, parts of speech, semantic tags and synsets -to track translationese. Our analysis shows that (i) semantic distances between original target language and translations into this target language can be detected using the notion of isomorphism, (ii) language family ties with characteristics similar to linguistically motivated phylogenetic trees can be inferred from the distances and (iii) with delexicalised embeddings exhibiting source-language interference most significantly, other levels of abstraction display the same tendency, indicating the lexicalised results to be not "just" due to possible topic differences between original and translated texts. To the best of our knowledge, this is the first time departures from isomorphism between embedding spaces are used to track translationese.

show abstract

“…Many of these approaches use language embeddings with sparse features encoding WALS feature values. Oncevay et al (2020) find that combining information from typological databases with embeddings learned during training of an NMT model can be beneficial for multilingual NMT.…”

Section: Typologically Informed Sharingmentioning

confidence: 96%

SIGTYP 2020 Shared Task: Prediction of Typological Features

Bjerva

Salesky²,

Mielke³

et al. 2020

Proceedings of the Second Workshop on Computational Research in Linguistic Typology

View full text Add to dashboard Cite

Typological knowledge bases (KBs) such as WALS (Dryer and Haspelmath, 2013) contain information about linguistic properties of the world's languages. They have been shown to be useful for downstream applications, including cross-lingual transfer learning and linguistic probing. A major drawback hampering broader adoption of typological KBs is that they are sparsely populated, in the sense that most languages only have annotations for some features, and skewed, in that few features have wide coverage. As typological features often correlate with one another, it is possible to predict them and thus automatically populate typological KBs, which is also the focus of this shared task. Overall, the task attracted 8 submissions from 5 teams, out of which the most successful methods make use of such feature correlations. However, our error analysis reveals that even the strongest submitted systems struggle with predicting feature values for languages where few features are known.

show abstract

Bridging Linguistic Typology and Multilingual Machine Translation with Multi-View Language Representations

Cited by 27 publications

References 39 publications

Family of Origin and Family of Choice: Massively Parallel Lexiconized Iterative Pretraining for Severely Low Resource Text-based Translation

Family of Origin and Family of Choice: Massively Parallel Lexiconized Iterative Pretraining for Severely Low Resource Text-based Translation

Understanding Translationese in Multi-view Embedding Spaces

SIGTYP 2020 Shared Task: Prediction of Typological Features

Contact Info

Product

Resources

About