Some Experiments with a Convex IBM Model 2

Abstract

Using a recent convex formulation of IBM Model 2, we propose a new initialization scheme which has some favorable comparisons to the standard method of initializing IBM Model 2 with IBM Model 1. Additionally, we derive the Viterbi alignment for the convex relaxation of IBM Model 2 and show that it leads to better F-Measure scores than those of IBM Model 2.

A Family of Latent Variable Convex Relaxations for IBM Model 2

“…However, for IBM Model 2, the data breaks such symmetries, so any relaxation will be nontrivial. (Simion, Collins, and Stein 2013) introduced the first convex relaxation of a model beyond IBM Model 1, design an algorithm for its optimization, and showed that it gives the same level of performance as IBM Model 2 (Simion, Collins, and Stein 2014).…”

“…For one, we could find the optimal alignment for I2CR-4 using IBM Model 2's rule (this is the optimal rule for I2CR-3 as well). On the other hand, using the same methods as presented in (Simion, Collins, and Stein 2014) we can find the optimal vector a (k) by splitting the maximization over the components of a (k) and focusing on finding a…”

“…Other IBM Models have non-concave objective functions with multiple local optima, and solving a non-convex problem to optimality is typically a computationally intractable task. Recently, using a linearization technique, a convex relaxation of IBM Model 2 was proposed (Simion, Collins, and Stein 2013;2014). In this work we generalize the methods introduced in (Simion, Collins, and Stein 2013) to yield a richer set of relaxation techniques.…”