Recursive Neural Network Based Preordering for English-to-Japanese Machine Translation

Abstract

The word order between source and target languages significantly influences the translation quality in machine translation. Preordering can effectively address this problem. Previous preordering methods require a manual feature design, making language dependent design costly. In this paper, we propose a preordering method with a recursive neural network that learns features from raw inputs. Experiments show that the proposed method achieves comparable gain in translation quality to the state-of-the-art method but without a manual feature design. 1 In this paper, we used binary syntax trees.

Neural Machine Translation with Reordering Embeddings

“…6) In Table 3, the +Pre-ordering performed worse that the baseline Transformer (base) for the WAT JA-EN translation task. We assume that the simple +pre-ordering strategy has negative impact on the translation performance of NMT model, which is in line with the functional similarity findings in (Du and Way, 2017;Kawara et al, 2018). Conversely, the proposed methods performed better than the Transformer (base), especially the +pre-ordering.…”

“…Pre-reordering, a pre-processing to make the source-side word orders close to those of the target side, has been proven very helpful for the SMT in improving translation quality. Moreover, neural networks were used to pre-reorder the sourceside word orders close to those of the target side (Du and Way, 2017;Kawara et al, 2018), and thus were input to the existing RNN-based NMT for improving the performance of translations. Du and Way (2017) and Kawara et al (2018) reported that the prereordering method had an negative impact on the NMT for the ASPEC JA-EN translation task.…”

“…Moreover, neural networks were used to pre-reorder the sourceside word orders close to those of the target side (Du and Way, 2017;Kawara et al, 2018), and thus were input to the existing RNN-based NMT for improving the performance of translations. Du and Way (2017) and Kawara et al (2018) reported that the prereordering method had an negative impact on the NMT for the ASPEC JA-EN translation task. In particular, Kawara et al (2018) assumed that one reason is the isolation between pre-ordering and NMT models, where both models are trained using independent optimization functions.…”

Preordering Encoding on Transformer for Translation

“…Rule-based preordering methods either manually create reordering rules [13]- [16] or extract reordering rules from a corpus [12], [20]. On the other hand, the references [3], [6], [21]- [23] applied machine learning to the preordering problem. Specifically, Hoshino et al [23] proposed a method that learns whether child nodes should be swapped at each node of a syntax tree.…”

“…An effective approach to address this issue is preordering, which reorders the words in a source sentence before it is translated. It is performed either through rule-based methods [4], [5], or by extracting the reordering rules automatically from a parallel corpus using machine learning-based methods [3], [6]. These methods improve translation quality in SMT, especially in case where the word orders in the source and target languages are highly dissimilar, such as between SVO and SOV languages.…”