Learning linear ordering problems for better translation

Abstract

We apply machine learning to the Linear Ordering Problem in order to learn sentence-specific reordering models for machine translation. We demonstrate that even when these models are used as a mere preprocessing step for German-English translation, they significantly outperform Moses' integrated lexicalized reordering model. Our models are trained on automatically aligned bitext. Their form is simple but novel. They assess, based on features of the input sentence, how strongly each pair of input word tokens w i , w j would like to reverse their relative order. Combining all these pairwise preferences to find the best global reordering is NP-hard. However, we present a non-trivial O(n 3) algorithm, based on chart parsing, that at least finds the best reordering within a certain exponentially large neighborhood. We show how to iterate this reordering process within a local search algorithm, which we use in training.

Learning Word Reorderings for Hierarchical Phrase-based Statistical Machine Translation

“…That means sub-models for reordering distance longer than a given threshold do not improve translation quality significantly. Compared with previous models (Tromble and Eisner, 2009;Feng et al, 2013), our method makes full use of helpful word reordering information and also avoids unnecessary computation cost for long distance reorderings. Besides, our reordering model is learned by feed-forward neural network (FNN) for better performance and uses efficient caching strategy to further reduce time cost.…”

“…Cui et al (2010) proposed a joint model to select hierarchical rules for both source and target sides. Hayashi et al (2010) demonstrated the effectiveness of using word reordering information within hierarchical phrase-based SMT by integrating Tromble and Eisner (2009)'s word reordering model into decoder as a feature, which estimates the probability of any two source words in a sentence being reordered during translating. Feng et al (2013) proposed a word reordering model to learn reorderings only for continuous words, which reduced computation cost a lot compared with Tromble and Eisner (2009)'s model and still achieved significant reordering improvement over the baseline system.…”

Source-side Preordering for Translation using Logistic Regression and Depth-first Branch-and-Bound Search

“…One such approach is to form a cascade of two translation systems, where the first one translates the source to its preordered version (Costa-jussà and Fonollosa, 2006). Alternatively, one can define models that assign a cost to the relative position of each pair of words in the sentence, and search for the sequence that optimizes the global score as a linear ordering problem (Tromble and Eisner, 2009) or as a traveling salesman problem (Visweswariah et al, 2011). Yet another line of work attempts to automatically induce a parse tree and a preordering model from word alignments (DeNero and Uszkoreit, 2011;Neubig et al, 2012).…”

“…Similarly to Yang et al (2012) we train a large discriminative linear model, but rather than model each child's position in an ordered list of children, we model a more natural pair-wise swap / no-swap preference (like Tromble and Eisner (2009) did at the word level). We then incorporate this model into a global, efficient branch-and-bound search through the space of permutations.…”

Syntax-Based Pre-reordering for Chinese-to-Japanese Statistical Machine Translation

“…Another direction of pre-reordering is to develop reordering rules without using a parser [74][75][76][77][78]. For instance, in [74], reordering source language was treated as a translation task in which statistical word classes were used; but in [75], reordering rules were learned from POS tags instead of parse trees; authors in [76] and [77] proposed methods of using binary classification; and Neubig et al [78] presented a traditional context-free-grammar models based method for learning a discriminative parser to improve reordering accuracy.…”