Neural Machine Translation with Source Dependency Representation

Chen, Kehai; Wang, Rui; Utiyama, Masao; Liu, Lemao; Tamura, Akihiro; Sumita, Eiichiro; Zhao, Tiejun

doi:10.18653/v1/d17-1304

Cited by 63 publications

(41 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Their work was further advanced in [5], [6], which encoded the entire source sentence. Chen et al [38], [43] flatten local dependency structures including parent, children, and sibling words into a word tuple, which is represeted as a compositonal vector by a convotional NN to model the long-distance dependency constraints. In spite of their success, their approaches center around capturing relations between each word and its local dependency words.…”

Section: Discussionmentioning

confidence: 99%

Syntax-Based Context Representation for Statistical Machine Translation

Chen

Zhao

Yang

2018

IEICE Trans. Inf. & Syst.

Self Cite

View full text Add to dashboard Cite

Learning semantic representation for translation context is beneficial to statistical machine translation (SMT). Previous efforts have focused on implicitly encoding syntactic and semantic knowledge in translation context by neural networks, which are weak in capturing explicit structural syntax information. In this paper, we propose a new neural network with a tree-based convolutional architecture to explicitly learn structural syntax information in translation context, thus improving translation prediction. Specifically, we first convert parallel sentences with source parse trees into syntax-based linear sequences based on a minimum syntax subtree algorithm, and then define a tree-based convolutional network over the linear sequences to learn syntax-based context representation and translation prediction jointly. To verify the effectiveness, the proposed model is integrated into phrase-based SMT. Experiments on large-scale Chinese-to-English and German-to-English translation tasks show that the proposed approach can achieve a substantial and significant improvement over several baseline systems. key words: syntax context representation, tree-based neural network, translation prediction, statistical machine translation † The "spurious" indicates that the source word is aligned to "NULL" of target sentence.

show abstract

Section: Discussionmentioning

confidence: 99%

Syntax-Based Context Representation for Statistical Machine Translation

Chen

Zhao

Yang

2018

IEICE Trans. Inf. & Syst.

Self Cite

View full text Add to dashboard Cite

show abstract

“…It has greatly improved the performance of translation. On this basis, there are many interesting and effective methods [5], [16], [26], [34], [35] which have been proposed in improving attention mechanism of the NMT system. Luong et al [5] proposed global attention model and local attention model, further compare several different scoring functions of the attention weight.…”

Section: Attention Mechanismmentioning

confidence: 99%

“…Tu et al [26] presented a coverage vector to keep track of the attention history and promote the attention mechanism to focus on more untranslated source words. Chen et al [16], [34] proposed a double context method by two attention mechanism to capture more source context information for translation prediction. Our work has the same source attention mechanism, compared with the above models, the forward and the reverse target attention are also imported, which can help to produce a more smooth translation.…”

Section: Attention Mechanismmentioning

confidence: 99%

Neural Machine Translation with Target-Attention Model

Yang

Zhang

Chen

et al. 2020

IEICE Trans. Inf. & Syst.

Self Cite

View full text Add to dashboard Cite

Attention mechanism, which selectively focuses on source-side information to learn a context vector for generating target words, has been shown to be an effective method for neural machine translation (NMT). In fact, generating target words depends on not only the source-side information but also the target-side information. Although the vanilla NMT can acquire target-side information implicitly by recurrent neural networks (RNN), RNN cannot adequately capture the global relationship between target-side words. To solve this problem, this paper proposes a novel target-attention approach to capture this information, thus enhancing target word predictions in NMT. Specifically, we propose three variants of target-attention model to directly obtain the global relationship among target words: 1) a forward target-attention model that uses a target attention mechanism to incorporate previous historical target words into the prediction of the current target word; 2) a reverse target-attention model that adopts a reverse RNN model to obtain the entire reverse target words information, and then to combine with source context information to generate target sequence; 3) a bidirectional target-attention model that combines the forward target-attention model and reverse target-attention model together, which can make full use of target words to further improve the performance of NMT. Our methods can be integrated into both RNN based NMT and self-attention based NMT, and help NMT get global target-side information to improve translation performance. Experiments on the NIST Chinese-to-English and the WMT English-to-German translation tasks show that the proposed models achieve significant improvements over state-of-the-art baselines.

show abstract

“…Operation sequence models are n-gram models that include lexical translation operations and reordering operations in a single generative story, thereby combining elements from the previous three model families (Durrani et al, 2011(Durrani et al, , 2013(Durrani et al, , 2014. Their method were further extended by source syntax information (Chen et al, 2017c(Chen et al, , 2018b to improve the performance of SMT.…”

Section: Reordering Model For Pbsmtmentioning

confidence: 99%

Neural Machine Translation with Reordering Embeddings

Chen¹,

Wang²,

Utiyama³

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

Self Cite

View full text Add to dashboard Cite

The reordering model plays an important role in phrase-based statistical machine translation. However, there are few works that exploit the reordering information in neural machine translation. In this paper, we propose a reordering mechanism to learn the reordering embedding of a word based on its contextual information. These reordering embeddings are stacked together with self-attention networks to learn sentence representation for machine translation. The reordering mechanism can be easily integrated into both the encoder and the decoder in the Transformer translation system. Experimental results on WMT'14 English-to-German, NIST Chinese-to-English, and WAT ASPEC Japanese-to-English translation tasks demonstrate that the proposed methods can significantly improve the performance of the Transformer translation system.

show abstract

Neural Machine Translation with Source Dependency Representation

Cited by 63 publications

References 16 publications

Syntax-Based Context Representation for Statistical Machine Translation

Syntax-Based Context Representation for Statistical Machine Translation

Neural Machine Translation with Target-Attention Model

Neural Machine Translation with Reordering Embeddings

Contact Info

Product

Resources

About