Gaussian Mixture Models with Concave Penalized Fusion

Abstract

Estimation of finite mixture models is a fundamental and challenging problem.We propose a penalized method for Gaussian mixture linear regression, where the error terms follow a location-scale mixture of Gaussian distributions. The objective function is a combination of the likelihood function of observed data and a penalty on pairwise differences of parameters. An alternating direction method of multipliers algorithm is developed, and its convergence property is established. By clustering and merging similar observations in an automatic manner, our method provides an integrated tool for simultaneously determining the number of components and estimating parameters in finite mixture models. Moreover, it allows the mean and precision parameters to have different structures, so that pooled estimators can be obtained. We also establish the statistical properties of our estimators. Extensive simulations and real data examples are presented to evaluate the numerical performance of the proposed method.