Bilingual Lexicon Induction through Unsupervised Machine Translation

Artetxe, Mikel; Labaka, Gorka; Agirre, Eneko

doi:10.18653/v1/p19-1494

Cited by 33 publications

(30 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…CLWE models-although similar in spirit to MONOTRANS-are only competitive on the easiest and smallest task (MLDoc), and perform poorly on the more challenging ones (XNLI and XQuAD). While previous work has questioned evaluation methods in this research area (Glavaš et al, 2019;Artetxe et al, 2019), our results provide evidence that existing methods are not competitive in challenging downstream tasks and that mapping between two fixed embedding spaces may be overly restrictive. For that reason, we think that designing better integration techniques of CLWE to downstream models is an important future direction.…”

Section: Transfer Of Monolingual Representationsmentioning

confidence: 54%

On the Cross-lingual Transferability of Monolingual Representations

Artetxe¹,

Ruder²,

Yogatama³

2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

Self Cite

437

412

View full text Add to dashboard Cite

State-of-the-art unsupervised multilingual models (e.g., multilingual BERT) have been shown to generalize in a zero-shot crosslingual setting. This generalization ability has been attributed to the use of a shared subword vocabulary and joint training across multiple languages giving rise to deep multilingual abstractions. We evaluate this hypothesis by designing an alternative approach that transfers a monolingual model to new languages at the lexical level. More concretely, we first train a transformer-based masked language model on one language, and transfer it to a new language by learning a new embedding matrix with the same masked language modeling objective-freezing parameters of all other layers. This approach does not rely on a shared vocabulary or joint training. However, we show that it is competitive with multilingual BERT on standard cross-lingual classification benchmarks and on a new Cross-lingual Question Answering Dataset (XQuAD). Our results contradict common beliefs of the basis of the generalization ability of multilingual models and suggest that deep monolingual models learn some abstractions that generalize across languages. We also release XQuAD as a more comprehensive cross-lingual benchmark, which comprises 240 paragraphs and 1190 question-answer pairs from SQuAD v1.1 translated into ten languages by professional translators.

show abstract

Section: Transfer Of Monolingual Representationsmentioning

confidence: 54%

On the Cross-lingual Transferability of Monolingual Representations

Artetxe¹,

Ruder²,

Yogatama³

2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

Self Cite

437

412

View full text Add to dashboard Cite

show abstract

“…Similarly, in addition to being a downstream application of the former, unsupervised machine translation can also be useful to develop other multilingual applications or learn better crosslingual representations. This has previously been shown for supervised machine translation (McCann et al, 2017;Siddhant et al, 2019) and recently for bilingual lexicon induction (Artetxe et al, 2019a). In light of these connections, we call for a more holistic view of UCL, both from an experimental and theoretical perspective.…”

Section: Bridging the Gap Between Unsupervised Cross-lingual Learning Flavorsmentioning

confidence: 51%

A Call for More Rigor in Unsupervised Cross-lingual Learning

Artetxe¹,

Ruder²,

Yogatama³

et al. 2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

Self Cite

View full text Add to dashboard Cite

We review motivations, definition, approaches, and methodology for unsupervised crosslingual learning and call for a more rigorous position in each of them. An existing rationale for such research is based on the lack of parallel data for many of the world's languages. However, we argue that a scenario without any parallel data and abundant monolingual data is unrealistic in practice. We also discuss different training signals that have been used in previous work, which depart from the pure unsupervised setting. We then describe common methodological issues in tuning and evaluation of unsupervised cross-lingual models and present best practices. Finally, we provide a unified outlook for different types of research in this area (i.e., cross-lingual word embeddings, deep multilingual pretraining, and unsupervised machine translation) and argue for comparable evaluation of these models.

show abstract

“…Instead of performing non-linear regression, which is less effective in a large dimension of data, they sample the source language and map them using kernel mapping. Artetxe et al [20] proposed an approach to build bilingual lexicon by generating synthetic parallel data. The synthetic parallel data is used as resource in unsupervised machine translation to generate bilingual dictionary.…”

Section: Related Work 21 Bilingual Lexicon Extraction From Comparablmentioning

confidence: 99%

Exploiting Comparable Corpora to Enhance Bilingual Lexicon Induction from Monolingual Corpora

Sholikah¹,

Morimoto²,

Arifin³

et al. 2020

IJIES

View full text Add to dashboard Cite

Bilingual lexicons are essential resources in natural language processing (NLP) and information retrieval (IR). Automatic bilingual lexicon acquisition relies on a large number of parallel corpora that can be scarce or even unavailable for several languages. On the other hand, there are other resources that can be used to build bilingual lexicon such as comparable corpora (aligned documents) and monolingual corpora that are easily to get and available in any language, including resource-limited languages. Hence, this paper proposes a two stages framework that can learn bilingual lexicons from monolingual corpora enhanced using comparable corpora without any additional resources. The framework consists of two stages: comparable dictionary building and monolingual mapping. Comparable dictionary building is a process to create coarse dictionary from comparable corpora by utilizing topic modeling approach. The second stage is monolingual mapping by using the result from the previous stage as seed initialization for the bi-directional projection learning. The utilization of comparable corpora can replace the need of bilingual dictionary. The experiment was conducted using three kinds of language pairs: English-®Indonesia, English-®Arabic and Arabic-®Indonesia. The result of the experiment showed that the proposed method can enhance the accuracy from monolingual corpora and outperform other previous methods.

show abstract

Bilingual Lexicon Induction through Unsupervised Machine Translation

Cited by 33 publications

References 24 publications

On the Cross-lingual Transferability of Monolingual Representations

On the Cross-lingual Transferability of Monolingual Representations

A Call for More Rigor in Unsupervised Cross-lingual Learning

Exploiting Comparable Corpora to Enhance Bilingual Lexicon Induction from Monolingual Corpora

Contact Info

Product

Resources

About