Non-parametric Mixture Models for Clustering

Abstract

Abstract. Mixture models have been widely used for data clustering. However, commonly used mixture models are generally of a parametric form (e.g., mixture of Gaussian distributions or GMM), which significantly limits their capacity in fitting diverse multidimensional data distributions encountered in practice. We propose a non-parametric mixture model (NMM) for data clustering in order to detect clusters generated from arbitrary unknown distributions, using non-parametric kernel density estimates. The proposed model is non-parametric since the generative distribution of each data point depends only on the rest of the data points and the chosen kernel. A leave-one-out likelihood maximization is performed to estimate the parameters of the model. The NMM approach, when applied to cluster high dimensional text datasets significantly outperforms the state-of-the-art and classical approaches such as K-means, Gaussian Mixture Models, spectral clustering and linkage methods.

Density Estimation in Several Populations With Uncertain Population Membership

“…In both cases, each observation is assumed to be drawn randomly from one of several populations. However, in clustering the probability density functions of the populations are often clearly differentiated (Mallapragada, Jin, and Jain 2010), whereas in our problem these functions do not even have to be different. In addition, in clustering, the likelihood that an observation belongs to a specific population is unknown, and identifying the population to which a particular observation belongs is of interest.…”

On the Sample Complexity of Random Fourier Features for Online Learning

“…2 Table I summarizes the statistics of these datasets. We use the Gaussian kernel with kernel width set to be 5th percentile of the pairwise distances [Mallapragada et al 2010;Abrahamsen and Hansen 2011]. The column σ is the kernel width for each dataset.…”

Heart Disease Diagnosis via Nonparametric Mixture Models

“…Unlike the parametric methods nonparametric density based clustering methods do not choose particular density function but normally use kernel density estimate with kernels one one's choice. The Gaussian kernel has so far proved to be the commonly used [12,13,14,15]. General researchers use the multidimensional kernel density estimator with adaptive bandwidth suggested by [16] and represented as equation (21):…”

“…Several authors have represented nonparametric mixture models in different ways with the general assumption that each cluster is generated by its own unknown density function [15]. Nonparametric mixture models can therefore also mean that no assumptions are made about the form of the density f j of model (1.1), even though the weights πj maybe scalar parameters as alluded by [17].…”