The Noise Component in Model-based Cluster Analysis

Abstract

Abstract. The so-called noise-component has been introduced by Banfield and Raftery (1993) to improve the robustness of cluster analysis based on the normal mixture model. The idea is to add a uniform distribution over the convex hull of the data as an additional mixture component. While this yields good results in many practical applications, there are some problems with the original proposal: 1) As shown by Hennig (2004), the method is not breakdown-robust.2) The original approach doesn't define a proper ML estimator, and doesn't have satisfactory asymptotic properties.We discuss two alternatives. The first one consists of replacing the uniform distribution by a fixed constant, modelling an improper uniform distribution that doesn't depend on the data. This can be proven to be more robust, though the choice of the involved tuning constant is tricky. The second alternative is to approximate the ML-estimator of a mixture of normals with a uniform distribution more precisely than it is done by the "convex hull" approach. The approaches are compared by simulations and for a real data example.

Adaptive Mixture Discriminant Analysis for Supervised Learning with Unobserved Classes

“…Good results of the use of this specific mixture model were observed in different situations. Hennig and Coretto [22] proposed recently to use an improper uniform distribution that does not depend on the data for improving the robustness and provide a better approximation of the likelihood than the one proposed in the original work. An application of noise detection is proposed in Section 4.…”

A Bayesian mixture model to quantify parameters of spatial clustering

“…While each cluster is generally represented by one mixture component, propositions have been recently made to model each cluster by several mixture components to give the model more flexibility (Baudry et al, 2010). Non-clustering observations, if considered, are generally viewed as noise rather than as a component of interest (see, e.g., Dasgupta and Raftery, 1998;Hennig and Coretto, 2008). A case can be made for fitting mixture models in a Bayesian framework (Frühwirth-Schnatter, 2006;Fritsch and Ickstadt, 2009).…”

Modelling Background Noise in Finite Mixtures of Generalized Linear Regression Models

“…, K the regular components. Banfield & Raftery (1993) and Hennig & Coretto (2007) use a uniform distribution for f 0 , the main difference is that the former estimate the range of the uniform from the data, while the latter use either an improper uniform with pre-specified fixed value for the height of the density, or an ML estimate for the complete mixture including the noise component. Both consider only the case of model-based clustering, i.e., no regression.…”

“…Thus, we knock out outliers everywhere except for the main support regions of the regular components. For uniforms, we need to solve the ill-conditioned estimation problem of the boundaries of the uniform distribution, see Hennig & Coretto (2007) for a detailed discussion. For the Gaussians exact estimation of variance is not really critical (a rather unusual situation!…”

“…A comprehensive theoretical analysis for breakdown points of ML-estimators of locationscale mixtures can be found in Hennig (2004). Suggested solutions for robustification against outliers include 1. to add a noise component which is either uniform over the convex hull of the complete data set (Banfield & Raftery (1993)), or an improper constant uniform (Hennig & Coretto (2007)), 2. replace Gaussian densities with t-densities (Mclachlan & Peel (2000)), and 3. trimming observations (Cuesta-Albertos et al (1997)). …”