End-to-End Neural Word Alignment Outperforms GIZA++

Zenkel, Thomas; Wuebker, Joern; DeNero, John

doi:10.18653/v1/2020.acl-main.146

Cited by 50 publications

(82 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To address this, Peter et al (2017) tailor attention matrices to obtain higher quality alignments. Li et al (2018)'s andZenkel et al (2019)'s models perform similarly to and Zenkel et al (2020) outperform Giza++. Ding et al (2019) propose better decoding algorithms to deduce word alignments from NMT predictions.…”

Section: Part-of-speech Analysismentioning

confidence: 94%

SimAlign: High Quality Word Alignments Without Parallel Training Data Using Static and Contextualized Embeddings

Sabet¹,

Dufter²,

Yvon³

et al. 2020

Findings of the Association for Computational Linguistics: EMNLP 2020

View full text Add to dashboard Cite

Word alignments are useful for tasks like statistical and neural machine translation (NMT) and cross-lingual annotation projection. Statistical word aligners perform well, as do methods that extract alignments jointly with translations in NMT. However, most approaches require parallel training data, and quality decreases as less training data is available. We propose word alignment methods that require no parallel data. The key idea is to leverage multilingual word embeddings -both static and contextualized -for word alignment. Our multilingual embeddings are created from monolingual data only without relying on any parallel data or dictionaries. We find that alignments created from embeddings are superior for four and comparable for two language pairs compared to those produced by traditional statistical aligners -even with abundant parallel data; e.g., contextualized embeddings achieve a word alignment F 1 for English-German that is 5 percentage points higher than eflomal, a high-quality statistical aligner, trained on 100k parallel sentences.

show abstract

Section: Part-of-speech Analysismentioning

confidence: 94%

SimAlign: High Quality Word Alignments Without Parallel Training Data Using Static and Contextualized Embeddings

Sabet¹,

Dufter²,

Yvon³

et al. 2020

Findings of the Association for Computational Linguistics: EMNLP 2020

View full text Add to dashboard Cite

show abstract

“…Opt. + Guided) in Zenkel et al (2020). We observe that SHIFT-AET performs better than BAO-GUIDED in terms of alignment accuracy.…”

Section: Alignment Resultsmentioning

confidence: 79%

“…Comparison with Zenkel et al (2020) Concurrent with our work, Zenkel et al (2020) propose a neural aligner that can outperform GIZA++. Table 3 compares the performance of SHIFT-AET and the best method BAO-GUIDED (Birdir.…”

Section: Alignment Resultsmentioning

confidence: 92%

“…Dataset We follow previous work (Zenkel et al, 2019(Zenkel et al, , 2020 in data setup and conduct experiments on publicly available datasets for German-English (de-en) 3 , Romanian-English (ro-en) and French-English (fr-en) 4 . Since no validation set is provided, we follow Ding et al (2019) to set the last 1,000 sentences of the training data before preprocessing as validation set.…”

Section: Settingsmentioning

confidence: 99%

“…To alleviate this problem, some researchers modify the transformer architecture by adding alignment modules that predict the to-be-aligned target token (Zenkel et al, 2019(Zenkel et al, , 2020 or modify the training loss by designing an alignment loss computed with full target sentence (Garg et al, 2019;Zenkel et al, 2020). Others argue that using only attention weights is insufficient for generating clean word alignment and propose to induce alignments with feature importance measures, such as leaveone-out measures (Li et al, 2019) and gradientbased measures (Ding et al, 2019).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Accurate Word Alignment Induction from Neural Machine Translation

Chen¹,

Liu²,

Chen³

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

Despite its original goal to jointly learn to align and translate, prior researches suggest that Transformer captures poor word alignments through its attention mechanism. In this paper, we show that attention weights DO capture accurate word alignments and propose two novel word alignment induction methods SHIFT-ATT and SHIFT-AET. The main idea is to induce alignments at the step when the to-be-aligned target token is the decoder input rather than the decoder output as in previous work. SHIFT-ATT is an interpretation method that induces alignments from the attention weights of Transformer and does not require parameter update or architecture change. SHIFT-AET extracts alignments from an additional alignment module which is tightly integrated into Transformer and trained in isolation with supervision from symmetrized SHIFT-ATT alignments. Experiments on three publicly available datasets demonstrate that both methods perform better than their corresponding neural baselines and SHIFT-AET significantly outperforms GIZA++ by 1.4-4.8 AER points. 1 * Corresponding author. Part of the work was done when Yun was in Huawei Noah's Ark Lab.1 Code can be found at https://github.com/ sufe-nlp/transformer-alignment.Source: das weiß ich . Dec. input: i understand this . Dec. output: i understand this .

show abstract