A Proximal Point Algorithm for Minimum Divergence Estimators with Application to Mixture Models

Abstract

Abstract:Estimators derived from a divergence criterion such as ϕ−divergences are generally more robust than the maximum likelihood ones. We are interested in particular in the so-called minimum dual ϕ-divergence estimator (MDϕDE), an estimator built using a dual representation of ϕ-divergences. We present in this paper an iterative proximal point algorithm that permits the calculation of such an estimator. The algorithm contains by construction the well-known Expectation Maximization (EM) algorithm. Our work is based on the paper of Tseng on the likelihood function. We provide some convergence properties by adapting the ideas of Tseng. We improve Tseng's results by relaxing the identifiability condition on the proximal term, a condition which is not verified for most mixture models and is hard to be verified for "non mixture" ones. Convergence of the EM algorithm in a two-component Gaussian mixture is discussed in the spirit of our approach. Several experimental results on mixture models are provided to confirm the validity of the approach.

A two‐step proximal‐point algorithm for the calculus of divergence‐based estimators in finite mixture models

“…Note that the verification of assumption A3 is a hard task, because it results in a set of n nonlinear equations in y i and cannot be treated in a similar way to the Gaussian mixture in Tseng () or Al Mohamad & Broniatowski ().…”

“…This procedure has the form of a proximal‐point algorithm and extends the EM algorithm. A similar algorithm was introduced in Al Mohamad & Broniatowski (). Here, in each iteration we have two steps: a step to calculate the proportion and a step to calculate the parameters of the mixture components.…”