A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings

Artetxe, Mikel; Labaka, Gorka; Agirre, Eneko

doi:10.18653/v1/p18-1073

Cited by 453 publications

(764 citation statements)

References 16 publications

Supporting

Mentioning

755

Contrasting

Unclassified

Order By: Relevance

“…Morphological GeneralizationWe propose a novel quadripartite analysis of the BLI models, in which we independently control for four different variables: (i) word form frequency, (ii) morphology, (iii) lexeme frequency and (iv)10 We also used our dictionaries to train and test the more recent model ofArtetxe et al (2018b) on a handful of languages and observed the same general trends.11 We did this within the FASTTEXT framework, using the trained .bin models for each of our 10 languages.…”

mentioning

confidence: 99%

Don’t Forget the Long Tail! A Comprehensive Analysis of Morphological Generalization in Bilingual Lexicon Induction

Czarnowska¹,

Ruder²,

Grave³

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

Human translators routinely have to translate rare inflections of words-due to the Zipfian distribution of words in a language. When translating from Spanish, a good translator would have no problem identifying the proper translation of a statistically rare inflection such as hablarámos. Note the lexeme itself, hablar, is relatively common. In this work, we investigate whether state-of-the-art bilingual lexicon inducers are capable of learning this kind of generalization. We introduce 40 morphologically complete dictionaries in 10 languages 1 and evaluate three of the state-of-the-art models on the task of translation of less frequent morphological forms. We demonstrate that the performance of state-of-the-art models drops considerably when evaluated on infrequent morphological inflections and then show that adding a simple morphological constraint at training time improves the performance, proving that the bilingual lexicon inducers can benefit from better encoding of morphology.

show abstract

mentioning

confidence: 99%

Don’t Forget the Long Tail! A Comprehensive Analysis of Morphological Generalization in Bilingual Lexicon Induction

Czarnowska¹,

Ruder²,

Grave³

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

show abstract

“…As table 7 shows, there are no improvements among above methods. Some linear mapping Approach EM F1 MUSE 33.03 49.48 DeMa (Zhou et al, 2019) 55.64 72.59 Vecmap (Artetxe et al, 2018) methods even causes devastating effect on EM/F1 scores.…”

Section: A23 Discussionmentioning

confidence: 99%

Zero-shot Reading Comprehension by Cross-lingual Transfer Learning with Multi-lingual Language Representation Model

Hsu¹,

Liu²,

Lee³

2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

Because it is not feasible to collect training data for every language, there is a growing interest in cross-lingual transfer learning. In this paper, we systematically explore zeroshot cross-lingual transfer learning on reading comprehension tasks with a language representation model pre-trained on multi-lingual corpus. The experimental results show that with pre-trained language representation zeroshot learning is feasible, and translating the source data into the target language is not necessary and even degrades the performance. We further explore what does the model learn in zero-shot setting 0 .

show abstract

“…The input of our method is a set of cross-lingual word embeddings and the monolingual corpora used to train them. In our experiments, we use fastText embeddings (Bojanowski et al, 2017) mapped through VecMap (Artetxe et al, 2018b), but the algorithm described next can also work with any other word embedding and cross-lingual mapping method.…”

Section: Proposed Methodsmentioning

confidence: 99%

“…The resulting cross-lingual embeddings are then used to induce the translations of words that were missing in the training dictionary by taking their nearest neighbor in the target language. The amount of required supervision was later reduced through self-learning methods (Artetxe et al, 2017), and then completely eliminated through adversarial training (Zhang et al, 2017a; or more robust iterative approaches combined with initialization heuristics (Artetxe et al, 2018b;Hoshen and Wolf, 2018). At the same time, several recent methods have formulated embedding mappings as an optimal transport problem (Zhang et al, 2017b;.…”

Section: Related Workmentioning

confidence: 99%

Bilingual Lexicon Induction through Unsupervised Machine Translation

Artetxe¹,

Labaka²,

Agirre³

2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

Self Cite

View full text Add to dashboard Cite

A recent research line has obtained strong results on bilingual lexicon induction by aligning independently trained word embeddings in two languages and using the resulting crosslingual embeddings to induce word translation pairs through nearest neighbor or related retrieval methods. In this paper, we propose an alternative approach to this problem that builds on the recent work on unsupervised machine translation. This way, instead of directly inducing a bilingual lexicon from cross-lingual embeddings, we use them to build a phrasetable, combine it with a language model, and use the resulting machine translation system to generate a synthetic parallel corpus, from which we extract the bilingual lexicon using statistical word alignment techniques. As such, our method can work with any word embedding and cross-lingual mapping technique, and it does not require any additional resource besides the monolingual corpus used to train the embeddings. When evaluated on the exact same cross-lingual embeddings, our proposed method obtains an average improvement of 6 accuracy points over nearest neighbor and 4 points over CSLS retrieval, establishing a new state-of-the-art in the standard MUSE dataset.

show abstract

A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings

Cited by 453 publications

References 16 publications

Don’t Forget the Long Tail! A Comprehensive Analysis of Morphological Generalization in Bilingual Lexicon Induction

Don’t Forget the Long Tail! A Comprehensive Analysis of Morphological Generalization in Bilingual Lexicon Induction

Zero-shot Reading Comprehension by Cross-lingual Transfer Learning with Multi-lingual Language Representation Model

Bilingual Lexicon Induction through Unsupervised Machine Translation

Contact Info

Product

Resources

About