Model-based Clustering with Dissimilarities: A Bayesian Approach

Abstract

A Bayesian model-based clustering method is proposed for clustering objects on the basis of dissimilarites. This combines two basic ideas. The first is that the objects have latent positions in a Euclidean space, and that the observed dissimilarities are measurements of the Euclidean distances with error. The second idea is that the latent positions are generated from a mixture of multivariate normal distributions, each one corresponding to a cluster. We estimate the resulting model in a Bayesian way using Markov chain Monte Carlo. The method carries out multidimensional scaling and model-based clustering simultaneously, and yields good object configurations and good clustering results with reasonable measures of clustering uncertainties. In the examples we study, the clustering results based on low-dimensional configurations were almost as good as those based on high-dimensional ones. Thus, the method can be used as a tool for dimension reduction when clustering high-dimensional objects, which may be useful especially for visual inspection of clusters.We also propose a Bayesian criterion for choosing the dimension of the object configuration and the number of clusters simultaneously. This is easy to compute and works reasonably well in simulations and real examples.

Solution path clustering with adaptive concave penalty

“…Spectral clustering techniques are based on graph theory and matrix decomposition and are gaining popularity as being simple, accurate, and able to find non-convex clusters (von Luxburg, 2007;Hastie, Tibshirani, and Friedman, 2009). Along with k-means and its variants, there also exist mixture-likelihood clustering approaches (McLachlan, Bean, and Peel, 2002;Fraley and Raftery, 2002;Yeung et al, 2001) that assume an underlying statistical model for the data and maximize the likelihood function with the EM algorithm or MCMC methods (Bensmail et al, 1997;Oh and Raftery, 2007). Yet other methods take modified or combined approaches, for example, self-organizing maps (Kohonen, 1990), dp-means (Kulis and Michael, 2011), CLICK (Sharan, Maron-Katz, and Shamir, 2003), gene shaving (Hastie et al, 2000), pclust (Wang, Neill, and Miller, 2008), support vector clustering (Ben-Hur et al, 2001) to name just a few.…”

Interpretable clustering using unsupervised binary trees

“…Recently, several new methods have been proposed for clustering analysis, see for instance Garcia Escudero et al [10] for a review with focus on robust clustering procedure. Other recent proposals have been made by Peña and Prieto ([18], [19]), Fraley and Raftery [7], Oh and Raftery [16], Walther [25], among others. But just a few different methods using decision trees to obtain clusters have previously been proposed.…”

A Latent Class Multidimensional Scaling Model for Two-Way One-Mode Continuous Rating Dissimilarity Data

“…Also, several models have been proposed in the literature to formulate latent class models for multidimensional scaling of paired comparison data (Formann, 1989;Böckenholt & Böckenholt, 1990;De Soete, 1990;De Soete & Winsberg, 1993a), pick any/n data (Böckenholt & Böckenholt, 1990, 1991De Soete & DeSarbo, 1991), multinomial choice data (Chintagunta, 1994), single stimulus preference data (DeSarbo, Howard, & Jedidi, 1991;DeSarbo et al, 1991;De Soete & Heiser, 1993;De Soete & Winsberg, 1993b), and three-way two mode data (Winsberg & De Soete, 1993), among others (see also DeSarbo, Ajay, and Lalita, 1994;DeSarbo et al, 1994;Wedel and DeSarbo, 1996;Andrews and Manrai, 1999, who also provide a good review of the literature). For two-way one-mode data, Oh and Raftery (2007) recently have proposed a Bayesian approximation based on a mixture of multivariate normal distributions for the latent class positions, assuming an inverse gamma, a Dirichlet, a normal and an inverse Wishart distribution as prior distributions for the parameters. 1 In this paper, we propose a latent class model for two-way one-mode continuous rating dissimilarity data that aims at partitioning the objects into classes and simultaneously estimating a low-dimensional spatial representation of T cluster points.…”