Generalizing Word Lattice Translation

Abstract

Word lattice decoding has proven useful in spoken language translation; we argue that it provides a compelling model for translation of text genres, as well. We extend lattice decoding to hierarchical phrase-based models, providing a unified treatment with phrase-based decoding by treating lattices as a case of weighted finite-state automata. In the process, we resolve a significant complication that lattice representations introduce in reordering models. Our experiments evaluating the approach demonstrate substantial gains for Chinese-English and Arabic-English translation.Keywords: word lattice translation, phrase-based and hierarchical models, statistical machine translation SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR'S ACRONYM(S) SPONSOR/MONITOR'S REPORT NUMBER(S) DISTRIBUTION/AVAILABILITY STATEMENTApproved for public release; distribution unlimited SUPPLEMENTARY NOTES ABSTRACTWord lattice decoding has proven useful in spoken language translation; we argue that it provides a compelling model for translation of text genres, as well. We extend lattice decoding to hierarchical phrase-based models, providing a unified treatment with phrase-based decoding by treating lattices as a case of weighted finite-state automata. In the process, we resolve a significant complication that lattice representations introduce in reordering models. Our experiments evaluating the approach demonstrate substantial gains for Chinese-English and Arabic-English translation. AbstractWord lattice decoding has proven useful in spoken language translation; we argue that it provides a compelling model for translation of text genres, as well. We extend lattice decoding to hierarchical phrase-based models, providing a unified treatment with phrase-based decoding by treating lattices as a case of weighted finite-state automata. In the process, we resolve a significant complication that lattice representations introduce in reordering models. Our experiments evaluating the approach demonstrate substantial gains for Chinese-English and Arabic-English translation.

Japanese Argument Reordering Based on Dependency Structure for Statistical Machine Translation

“…Therefore, we do not think that lattice decoding is suitable for our approach. Since Octavian does not provide a lattice decoding function, we rebuilt a translation model using Moses [25], [26] with the same configuration. We did not incorporate any arc probability for the lattice besides the weight of the branches.…”

Optimizing Tokenization Choice for Machine Translation across Multiple Target Languages

“…Their model, however, didn't exhibit any significant BLEU enhancement. One of the notable contributions within this domain is the work of Dyer et al (2008) and (Dyer, 2009), where they use a word lattice that encodes the surface forms (unsegmented words) as an option, and the full morphological breakdown of the surface form as another option. In this scope, the lattice is used to model a back-off system for the full morphological segmentation, rather than encoding the various tokenization schemes.…”

“…We use a similar approach to Dyer's (Dyer et al, 2008) for lattice-based decoding of tokenization options, but through encoding all tokenization options at the lattice instead of using it as a backoff model to full morphological breakdown as they use it.…”

“…The word lattice decoding follows the noisier channel model (Dyer et al, 2008). Word lattices are primarily used to model ambiguity in NLP systems, this ambiguity can be referred to by an observed ambiguity signal, which produces a set of source-language strings f ′ ∈ F(s).…”

Getting Past the Language Gap: Innovations in Machine Translation

“…The core of the framework is a transfer engine using two language-pair-dependent resources: a grammar of weighted synchronous context-free rules, and a probabilistic bilingual lexicon. Once the resources have been provided, the Stat-XFER framework carries out translation in a two-stage process, first applying the lexicon and grammar to parse synchronously an input sentence, then running a monotonic decoder over the resulting lattice of scored translation pieces assembled during parsing to produce a final string output (see Dyer et al 2008). Reordering is applied only in the first stage, driven by the syntactic grammar; the second-stage monotonic decoder only assembles translation fragments into complete hypotheses.…”