Estimation of Finite Mixture Distributions Through Bayesian Sampling

Abstract

A formal Bayesian analysis of a mixture model usually leads to intractable calculations, since the posterior distribution takes into account all the partitions of the sample. We present approximation methods which evaluate the posterior distribution and Bayes estimators by Gibbs sampling, relying on the missing data structure of the mixture model. The data augmentation method is shown to converge geometrically, since a duality principle transfers properties from the discrete missing data chain to the parameters. The fully conditional Gibbs alternative is shown to be ergodic and geometric convergence is established in the normal case. We also consider non-informative approximations associated with improper priors, assuming that the sample corresponds exactly to a k-component mixture.

A multiple imputation approach to deal with the unity measure error

“…The data augmentation algorithm for Bayesian estimation of mixture models is studied by Diebolt and Robert (1994). In this case, the missing variables are the allocation variables Z i .…”

A new test for the presence of a normal mixture distribution based on the posterior Bayes factor

“…This characteristic feature of the Bayes factor greatly complicates its routine application, as in general the diffuse prior densities have to be replaced by conjugate or other prior densities depending on hyperparameters; variations in the values of the hyperparameters may seriously affect the value of the Bayes factor (see Aitkin, 1991, andDebolt andRobert, 1994, for other discussion). The posterior mean of the likelihood does not depend on the diffuse prior constant, and is insensitive to variations in the hyperparameters even if conjugate priors are used (Aitkin, 1991).…”

“…For larger sample sizes other computational methods for the posterior Bayes factor would be required; methods based on the Gibbs sampler have been suggested by Diebolt and Robert (1994).…”

A Bayesian predictive approach to determining the number of components in a mixture distribution

“…First we simulated 50 observations from 0.36 • N(lll, 529) + 0.64 x N(190, 324) as given in Diebolt and Robert (1991). As mentioned earlier we set ~j = 0.5, s 2 = 0.000002, ~,j = 3.0 for all j to have an approximate non-informative prior selection.…”

“…Diebolt and Robert (1994) study sampling-based approaches to approximating the Bayes estimates for finite mixtures of normal Dey, Kuo and Sahu distributions assuming the number of components k is known. Crawford (1991) proposes a modification of the Laplace method to estimate the Bayes estimators.…”