Pushdown Automata in Statistical Machine Translation

Abstract

This article describes the use of pushdown automata (PDA) in the context of statistical machine translation and alignment under a synchronous context-free grammar. We use PDAs to compactly represent the space of candidate translations generated by the grammar when applied to an input sentence. General-purpose PDA algorithms for replacement, composition, shortest path, and expansion are presented. We describe HiPDT, a hierarchical phrase-based decoder using the PDA representation and these algorithms. We contrast the complexity of this decoder with a decoder based on a finite state automata representation, showing that PDAs provide a more suitable framework to achieve exact decoding for larger synchronous context-free grammars and smaller language models. We assess this experimentally on a large-scale Chinese-to-English alignment and translation task. In translation, we propose a two-pass decoding strategy involving a weaker language model in the first-pass to address the results of PDA complexity analysis. We study in depth the experimental conditions and tradeoffs in which HiPDT can achieve state-of-the-art performance for large-scale SMT.

Syntactically Guided Neural Machine Translation

“…A target language sentence y can be a translation of a source language sentence x if there is a derivation D in the grammar which yields both y and x: y = y(D), x = x(D). This defines a regular language Y over strings in the target language via a projection of the sentence to be translated: Y = {y(D) : x(D) = x} (Iglesias et al, 2011;Allauzen et al, 2014). Scores are defined over derivations via a log-linear model with features {φ i } and weights λ.…”

Transducer Disambiguation with Sparse Topological Features

“…Lexicographic semirings, used for PoS tagging disambiguation , have been also shown to be useful in other tasks (Sproat et al, 2014), such as optimized epsilon encoding for backoff language models , and hierarchical phrase-based decoding with Pushdown Automata (Allauzen et al, 2014).…”

SGNMT – A Flexible NMT Decoding Platform for Quick Prototyping of New Models and Search Strategies

“…SGNMT is based on the OpenFSTbased Cambridge SMT system (Allauzen et al, 2014). Although the system is less than a year old, we have found it to be very flexible and easy for new researchers to adopt.…”

“…The strict separation of scoring module and search strategy and the decoupling of scoring modules from each other makes SGNMT a very flexible decoding tool for neural and symbolic models which is applicable not only to machine translation. SGNMT is based on the OpenFSTbased Cambridge SMT system (Allauzen et al, 2014). Although the system is less than a year old, we have found it to be very flexible and easy for new researchers to adopt.…”