Gromov-Wasserstein Alignment of Word Embedding Spaces

Alvarez-Melis, David; Jaakkola, Tommi S.

doi:10.18653/v1/d18-1214

Cited by 220 publications

(201 citation statements)

References 29 publications

Supporting

Mentioning

200

Contrasting

Order By: Relevance

“…Furthermore, there is a surprising lack of testing BLI results for statistical significance. The mismatches in evaluation yield partial conclusions and inconsistencies: on the one hand, some unsupervised models (Artetxe et al, 2018b;Hoshen and Wolf, 2018;Alvarez-Melis and Jaakkola, 2018) are reported to outperform (or at least perform on par with) previous best-performing supervised CLE models (Artetxe et al, 2017;Smith et al, 2017). On the other hand, the most recent supervised approaches (Doval et al, 2018; report further performance gains, surpassing unsupervised models.…”

Section: Introductionmentioning

confidence: 99%

How to (Properly) Evaluate Cross-Lingual Word Embeddings: On Strong Baselines, Comparative Analyses, and Some Misconceptions

Glavaš¹,

Litschko²,

Ruder³

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

142

137

View full text Add to dashboard Cite

Cross-lingual word embeddings (CLEs) enable multilingual modeling of meaning and facilitate cross-lingual transfer of NLP models. Despite their ubiquitous usage in downstream tasks, recent increasingly popular projectionbased CLE models are almost exclusively evaluated on a single task only: bilingual lexicon induction (BLI). Even BLI evaluations vary greatly, hindering our ability to correctly interpret performance and properties of different CLE models. In this work, we make the first step towards a comprehensive evaluation of cross-lingual word embeddings. We thoroughly evaluate both supervised and unsupervised CLE models on a large number of language pairs in the BLI task and three downstream tasks, providing new insights concerning the ability of cutting-edge CLE models to support cross-lingual NLP. We empirically demonstrate that the performance of CLE models largely depends on the task at hand and that optimizing CLE models for BLI can result in deteriorated downstream performance. We indicate the most robust supervised and unsupervised CLE models and emphasize the need to reassess existing baselines, which still display competitive performance across the board. We hope that our work will catalyze further work on CLE evaluation and model analysis.

show abstract

Section: Introductionmentioning

confidence: 99%

How to (Properly) Evaluate Cross-Lingual Word Embeddings: On Strong Baselines, Comparative Analyses, and Some Misconceptions

Glavaš¹,

Litschko²,

Ruder³

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

142

137

View full text Add to dashboard Cite

show abstract

“…Still, one should keep in mind that it has also been shown that there can be stability problems with SHAP. 417 …”

Section: How To Interpret the Results: Avoiding The Clever Hansmentioning

confidence: 99%

“…Thanks to libraries like tf-explain 420 and keras-vis, 421 appealing visualizations of model explanations are often only one function call away, but one should be aware that there are many caveats wherefore some sanity checks (such as randomization tests or addition of noise) should be used before relying on such a model interpretation. 417 , 422…”

Section: How To Interpret the Results: Avoiding The Clever Hansmentioning

confidence: 99%

Big-Data Science in Porous Materials: Materials Genomics and Machine Learning

et al. 2020

View full text Add to dashboard Cite

By combining metal nodes with organic linkers we can potentially synthesize millions of possible metal–organic frameworks (MOFs). The fact that we have so many materials opens many exciting avenues but also create new challenges. We simply have too many materials to be processed using conventional, brute force, methods. In this review, we show that having so many materials allows us to use big-data methods as a powerful technique to study these materials and to discover complex correlations. The first part of the review gives an introduction to the principles of big-data science. We show how to select appropriate training sets, survey approaches that are used to represent these materials in feature space, and review different learning architectures, as well as evaluation and interpretation strategies. In the second part, we review how the different approaches of machine learning have been applied to porous materials. In particular, we discuss applications in the field of gas storage and separation, the stability of these materials, their electronic properties, and their synthesis. Given the increasing interest of the scientific community in machine learning, we expect this list to rapidly expand in the coming years.

show abstract

“…If the weights of alignment are not important, the neural network without attention mechanism may also effectively detect parallel sentences since all alignments have the same contribution. However, the alignment deeply depends on linguistics and context [ 23 – 25 ]. For example, the English word “bearing” means multiple Chinese words such as “chengzhou,” “baochi,” and “zhoucheng” in a different context.…”

Section: Resultsmentioning

confidence: 99%

Extracting Parallel Sentences from Nonparallel Corpora Using Parallel Hierarchical Attention Network

Zhu

Yang

Chun

2020

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Collecting parallel sentences from nonparallel data is a long-standing natural language processing research problem. In particular, parallel training sentences are very important for the quality of machine translation systems. While many existing methods have shown encouraging results, they cannot learn various alignment weights in parallel sentences. To address this issue, we propose a novel parallel hierarchical attention neural network which encodes monolingual sentences versus bilingual sentences and construct a classifier to extract parallel sentences. In particular, our attention mechanism structure can learn different alignment weights of words in parallel sentences. Experimental results show that our model can obtain state-of-the-art performance on the English-French, English-German, and English-Chinese dataset of BUCC 2017 shared task about parallel sentences’ extraction.

show abstract

Gromov-Wasserstein Alignment of Word Embedding Spaces

Cited by 220 publications

References 29 publications

How to (Properly) Evaluate Cross-Lingual Word Embeddings: On Strong Baselines, Comparative Analyses, and Some Misconceptions

How to (Properly) Evaluate Cross-Lingual Word Embeddings: On Strong Baselines, Comparative Analyses, and Some Misconceptions

Big-Data Science in Porous Materials: Materials Genomics and Machine Learning

Extracting Parallel Sentences from Nonparallel Corpora Using Parallel Hierarchical Attention Network

Contact Info

Product

Resources

About