Neural Machine Translation between Vietnamese and English: an Empirical Study

Abstract

Machine translation is shifting to an end-to-end approach based on deep neural networks. The state of the art achieves impressive results for popular language pairs such as English -French or English -Chinese. However for English -Vietnamese the shortage of parallel corpora and expensive hyper-parameter search present practical challenges to neural-based approaches. This paper highlights our efforts on improving English-Vietnamese translations in two directions: (1) Building the largest open Vietnamese -English corpus to date, and (2) Extensive experiments with the latest neural models to achieve the highest BLEU scores. Our experiments provide practical examples of effectively employing different neural machine translation models with low-resource language pairs. IntroductionMachine translation is shifting to an end-to-end approach based on deep neural networks. Recent studies in neural machine translation (NMT) such as [41,2,42,14] have produced impressive advancements over phrase-based systems while eliminating the need for handengineered features. Most NMT systems are based on the encoder-decoder architecture which consists of two neural networks. The encoder compresses the source sequences into a real-valued vector, which is consumed by the decoder to generate the target sequences. The process is done in an end-to-end fashion, demonstrated the capability of learning representation directly from the training data.The typical sequence-to-sequence machine translation model consists of two recurrent neural networks (RNNs) and an attention mechanism [2,26]. Despite great improvements over traditional models [42,35,27] this architecture has certain shortcomings, namely that the recurrent networks are not easily parallelized and limited gradient flow while training deep models.Recent designs such as ConvS2S [14] and Transformer [41] can be better parallelized while producing better results on WMT datasets. However, NMT models take a long time to train and include many hyper-parameters. There is a number of works that tackle the problem of hyper-parameter selection [5,33] but they mostly focus on high-resource language pairs data, thus their findings may not translate well to low-resource translation tasks such as English-Vietnamese. Unlike in Computer Vision [17,20], the task of adapting parameters spaces from one NMT model to other NMT models is nearly impossible [5]. This reason limits researchers and engineers to reach good-chose hyper-parameters and well-trained models.

Improving neural machine translation by integrating transliteration for low-resource English–Assamese language

“…In the decoder, the hidden state s t is computed using the previous decoder hidden state s t−1 , the previous decoder output y t−1 , and the context vector c. It is represented as -Vu et al 2019), where E A is an activation function.…”

A Study on Error Feature Analysis and Error Correction in English Translation Through Machine Translatio

“…The second evaluation indicator used was BLEU [17], which is very common in the evaluation metrics of machine translation, and it was used to evaluate the difference values in the model-generated machine translation text and the actual correct text. Its value was between 0 and 1.…”

Query Translation Optimization and Mathematical Modeling for English-Chinese Cross-Language Information Retrieval

“…The English-Chinese translation mathematical model constructed in this subject has been built with componentized thinking. The advantage of this thinking is that it can measure the role of each component during the completion of the translation function from a better perspective [3]. When the model is trained, the standard training method can be used, and the typical training method can measure each component.…”