Bi-Decoder Augmented Network for Neural Machine Translation

Abstract

Neural Machine Translation (NMT) has become a popular technology in recent years, and the encoderdecoder framework is the mainstream among all the methods. It's obvious that the quality of the semantic representations from encoding is very crucial and can significantly affect the performance of the model. However, existing unidirectional source-to-target architectures may hardly produce a language-independent representation of the text because they rely heavily on the specific relations of the given language pairs. To alleviate this problem, in this paper, we propose a novel Bi-Decoder Augmented Network (BiDAN) for the neural machine translation task. Besides the original decoder which generates the target language sequence, we add an auxiliary decoder to generate back the source language sequence at the training time. Since each decoder transforms the representations of the input text into its corresponding language, jointly training with two target ends can make the shared encoder has the potential to produce a language-independent semantic space. We conduct extensive experiments on several NMT benchmark datasets and the results demonstrate the effectiveness of our proposed approach.

Recent Progress, Emerging Techniques, and Future Research Prospects of Bangla Machine Translation: A Systematic Review

“…Gated recurrent unit (GRU) employment of [91] and parts of speech tagging of [92] in attention mechanism might also be useful to employ in Bangla MT. In the line of data augmentation, input denoising plus auxiliary decoder investigated in [93] and selflearning, training with synthetically generated data using monolingual a source language corpus, investigated in [94], are also intuitive to improve MT performance for a lowresource language like Bangla. Multi-source translation, an approach to exploit multiple inputs (e.g., in two different languages) to increase performance, and missing data management investigated by Nishimura et al [95] might also be a way to achieve better Bangla MT performance.…”