Hybrid Machine Translation with Multi-Source Encoder-Decoder Long Short-Term Memory in English-Malay Translation

Abstract

Statistical Machine Translation (SMT) and Neural Machine Translation (NMT) are the state-of-the-art approaches in machine translation (MT). The translation produced by an SMT is based on the statistical analysis of text corpora, while NMT uses the deep neural network to model and to generate a translation. SMT and NMT have their strength and weaknesses. SMT may produce a better translation with a small parallel text corpus compared to NMT. Nevertheless, when the amount of parallel text available is large, the quality of the translation produced by NMT is often higher than SMT. Besides that, study also shown that the translation produced by SMT is better than NMT in cases where there is a domain mismatch between training and testing. SMT also has an advantage in long sentences. In addition, when a translation produced by an NMT is wrong, it is very difficult to find the error. In this paper, we investigate a hybrid approach that combines SMT and NMT to perform English to Malay translation. The motivation for using a hybrid machine translation is to combine the strength of both approaches to produce a more accurate translation. Our approach uses the multi-source encoder-decoder long short-term memory (LSTM) architecture. The architecture uses two encoders, one to embed the sentence to be translated, and another encoder to embed the initial translation produced by SMT. The translation from the SMT can be viewed as a "suggestion translation" to the neural MT. Our experiments show that the hybrid MT increases the BLEU scores of our best baseline machine translation in the computer science domain and news domain from 21.21 and 48.35 to 35.97 and 61.81 respectively.

A Spatiotemporal Convolutional Gated Recurrent Unit Network for Mean Wave Period Field Forecasting

“…However, RNN suffers from the problems of gradient explosion and disappearance. In the past few years, the long short-term memory (LSTM) networks with obvious advantages have shown record performance on various tasks, such as image processing [25], language translation [26,27], and handwriting recognition [28,29].…”

Sequence-to-sequence neural machine translation for English-Malay

“…As a type of natural language process (NLP) application [4], MT involves the process of using computer software to translate messages from a specific language into another language [5], [6], [7]. This process involves a source natural language (e.g., English) and a target natural language (e.g.…”

“…This is the essential process [8] in news translation, movie subtitling, question/answer systems and chatbots with understanding of different languages [9]. Two common state-of-the art approaches [5] are statistical machine translation (SMT) and neural machine translation (NMT) [10], [11].…”

“…NMT depends on a word embedding to transform the word sequence into a vector before the model training can take place [17]. Besides that, there are some work done by combining both SMT and NMT to take advantage of the strength both models [5]. Some of these works include Stahlberg et al that used risk estimation in NMT [18] and Du and Way's cascade framework in the hybrid MT [19].…”