Improving Statistical Machine Translation Using Bayesian Word Alignment and Gibbs Sampling

In word alignment certain source words are only needed for fluency reasons and do not have a translation on the target side. Most word alignment models assume a target NULL word from which they generate these untranslatable source words. Hypothesising a target NULL word is not without problems, however. For example, because this NULL word has a position, it interferes with the distribution over alignment jumps. We present a word alignment model that accounts for untranslatable source words by generating them from preceding source words. It thereby removes the need for a target NULL word and only models alignments between word pairs that are actually observed in the data. Translation experiments on English paired with Czech, German, French and Japanese show that the model outperforms its traditional IBM counterparts in terms of BLEU score.

show abstract

“…We compare our model and the Bayesian IBM models 1 and 2 of Mermer et al (2013) against IBM model 2 as a baseline.…”

Section: Experiments and Resultsmentioning

confidence: 99%

“…Derivations of samplers similar to ours can be found in the appendices of Mermer et al (2013) and Griffiths et al (2007). We omit the derivation here for space reasons.…”

Section: Inferencementioning

confidence: 99%

Word Alignment without NULL Words

Schulz¹,

Aziz²,

Sima’an³

2016

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

View full text Add to dashboard Cite

show abstract

“…These constraints have recently been modeled with sparse and symmetric Dirichlet priors (Mermer and Saraçlar, 2011;Mermer et al, 2013;Riley and Gildea, 2012) which, beyond capturing the range of lexical distributions we consider likely, also turn out to be mathematically very convenient as the Dirichlet distribution is a conjugate prior to the categorical distribution. The d-dimensional Dirichlet distribution is defined over the space of d-dimensional categorical distributions, and is parameterized by the d-…”

Section: Bayesian Ibm Modelsmentioning

confidence: 99%

“…Recently, several authors have disposed with EM altogether, relying entirely on Gibbs sampling for inference in IBM-based models with Bayesian priors of varying complexity (Mermer and Saraçlar, 2011;Mermer et al, 2013;Gal and Blunsom, 2013;Östling, 2015). Of these, Gal and Blunsom (2013) and to some extent Östling (2015) prioritize maximizing alignment accuracy, which is obtained by using complex hierarchical models.…”

Section: Related Workmentioning

confidence: 99%

“…Of these, Gal and Blunsom (2013) and to some extent Östling (2015) prioritize maximizing alignment accuracy, which is obtained by using complex hierarchical models. Mermer et al (2013) use Dirichlet priors with IBM models 1 and 2 to obtain efficient samplers, which they implement in an approximate fashion (where dependencies between variables are ignored during sampling) in order to facilitate parallelization. This article follows previous work by the first author (Östling, 2015), which however was focused on alignment of short parallel text for applications in language typology and transfer learning, rather than efficient large-scale alignment for use with statistical machine translation systems.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Efficient Word Alignment with Markov Chain Monte Carlo

Östling

Tiedemann

2016

The Prague Bulletin of Mathematical Linguistics

View full text Add to dashboard Cite

We present efmaral, a new system for efficient and accurate word alignment using a Bayesian model with Markov Chain Monte Carlo (MCMC) inference. Through careful selection of data structures and model architecture we are able to surpass the fast_align system, commonly used for performance-critical word alignment, both in computational efficiency and alignment accuracy. Our evaluation shows that a phrase-based statistical machine translation (SMT) system produces translations of higher quality when using word alignments from efmaral than from fast_align, and that translation quality is on par with what is obtained using giza++, a tool requiring orders of magnitude more processing time. More generally we hope to convince the reader that Monte Carlo sampling, rather than being viewed as a slow method of last resort, should actually be the method of choice for the SMT practitioner and others interested in word alignment.

show abstract

Improving Word Alignment with Contextualized Embedding and Bilingual Dictionary

Yan

2021

Communications in Computer and Information Science

View full text Add to dashboard Cite

Improving Statistical Machine Translation Using Bayesian Word Alignment and Gibbs Sampling

Cited by 11 publications

References 26 publications

Word Alignment without NULL Words

Word Alignment without NULL Words

Efficient Word Alignment with Markov Chain Monte Carlo

Improving Word Alignment with Contextualized Embedding and Bilingual Dictionary

Contact Info

Product

Resources

About