Coverage Embedding Models for Neural Machine Translation

Abstract

In this paper, we enhance the attention-based neural machine translation (NMT) by adding explicit coverage embedding models to alleviate issues of repeating and dropping translations in NMT. For each source word, our model starts with a full coverage embedding vector to track the coverage status, and then keeps updating it with neural networks as the translation goes. Experiments on the large-scale Chinese-to-English task show that our enhanced model improves the translation quality significantly on various test sets over the strong large vocabulary NMT system.

Cutting-off Redundant Repeating Generations for Neural Abstractive Summarization

“…coverage (Tu et al, 2016;Mi et al, 2016;. This is why we say upper-bound frequency estimation, not just (exact) frequency.…”

“…This issue has been discussed in the neural MT (NMT) literature as a part of a coverage problem (Tu et al, 2016;Mi et al, 2016). Such repeating generation behavior can become more severe in some NLG tasks than in MT.…”

The QT21 Combined Machine Translation System for English to Latvian

“…Second, we strengthen the attentional network with a coverage vector accumulating the previous attentional information, similar to the work of Mi et al (2016) and Tu et al (2016b).…”

Detecting Untranslated Content for Neural Machine Translation

“…For comparison, we used the baseline NMT system with soft coverage models (Mi et al, 2016;Tu NTCIR-10 , 2016b), which were used in first-pass decoding. 11 Whereas these studies used gated recurrent units (GRUs) (Chung et al, 2014) for the NMT and coverage models, we used LSTM.…”

“…We introduced soft coverage models (Tu et al, 2016b;Mi et al, 2016) in Section 1. In addition to these published studies, there are several parallel related studies on arXiv (Wu et al, 2016;Li and Jurafsky, 2016;Tu et al, 2016a).…”