Dynamic Past and Future for Neural Machine Translation

Abstract

Previous studies have shown that neural machine translation (NMT) models can benefit from explicitly modeling translated (PAST) and untranslated (FUTURE) source contents as recurrent states (Zheng et al., 2018). However, this less interpretable recurrent process hinders its power to model the dynamic updating of PAST and FUTURE contents during decoding. In this paper, we propose to model the dynamic principles by explicitly separating source words into groups of translated and untranslated contents through parts-to-wholes assignment. The assignment is learned through a novel variant of routing-by-agreement mechanism (Sabour et al., 2017), namely Guided Dynamic Routing, where the translating status at each decoding step guides the routing process to assign each source word to its associated group (i.e., translated or untranslated content) represented by a capsule, enabling translation to be made from holistic context. Experiments show that our approach achieves substantial improvements over both RNMT and Transformer by producing more adequate translations. Extensive analysis demonstrates that our method is highly interpretable, which is able to recognize the translated and untranslated contents as expected. 1

Acquiring Knowledge from Pre-Trained Model to Neural Machine Translation

“…Translation Quality The results on the EN→DE, DE→EN and ZH→EN are shown in Table 1. For a fair comparison, we also report several Transformer baseline from previous work (Vaswani et al 2017;Zheng et al 2019;Dou et al 2018). Our Transformer baseline achieves similar or better results comparing with them.…”

Towards Enhancing Faithfulness for Neural Machine Translation

“…Several Transformer systems with the same settings (Vaswani et al, 2017;Hassan et al, 2018;Gu et al, 2017) are reported as a comparison (line 1-6). Then, several related researches about improve faithfulness of NMT (Kong et al, 2019;Zheng et al, 2019;Feng et al, 2020) or exploiting translations for improving NMT (Xia et al, 2017;) also be reported (line 7-12). We implement three comparable approaches on our Transformer baseline, including: 1).…”

“…proposed to model global representation in the source side to improve the source representation. Zheng et al (2019) proposed a capsule based module to control the source representation dynamically in the decoding process. ), Feng et al (2020 and Garg et al (2019) proposed to introduce word alignment information in Transformer to improve translation accuracy.…”

“…Several recent studies are proposed following one of the above perspectives and have achieved considerable effects. Zheng et al (2019) proposed to divide the encoder output into past and future parts to fine-grained modeling contextual representation. Feng et al (2020) proposed a faithfulness part to optimize the contextual representation before feeding into the decoder.…”

Revisit Automatic Error Detection for Wrong and Missing Translation – A Supervised Approach

“…Therefore, we construct a high quality annotated corpus (TransErr) comprising 15000 Chinese-English translation pairs with inter-annotator agreement at 0.804 measured by Cohen's Kappa (Cohen, 1960). Different from existing error detection works which focus on all error classes, we currently only take care of missing and wrong translation , the major errors related to adequacy, which is a wide-known issue in neural machine translation (NMT) (Zheng et al, 2019). The errors tags are annotated on source (Chinese) sentences to reflect the loyalty and adequacy with respect to the source.…”