Large-scale Reordering Model for Statistical Machine Translation using Dual Multinomial Logistic Regression

Abstract

Phrase reordering is a challenge for statistical machine translation systems. Posing phrase movements as a prediction problem using contextual features modeled by maximum entropy-based classifier is superior to the commonly used lexicalized reordering model. However, Training this discriminative model using large-scale parallel corpus might be computationally expensive. In this paper, we explore recent advancements in solving large-scale classification problems. Using the dual problem to multinomial logistic regression, we managed to shrink the training data while iterating and produce significant saving in computation and memory while preserving the accuracy.

Stability of the Stochastic Gradient Method for an Approximated Large Scale Kernel Machine

“…We numerically showed the effect of three parameters; model size, number of Fourier components and learning rate choices on the stability. Table (1) shows the description of four binary classification datasets used for the analysis. These datasets can be downloaded from UCI machine learning repository website.…”

“…SGM computes the estimates of the gradient on the basis of a single randomly chosen sample in each iteration. Therefore, applying a stochastic gradient method for large scale machine learning problems can be computationally efficient [22,1,3,11]. In the context of supervised learning, models that are trained by such iterative optimization algorithms are commonly controlled by convergence rate analysis.…”