Clustering by Passing Messages Between Data Points

Abstract

Clustering data by identifying a subset of representative examples is important for processing sensory signals and detecting patterns in data. Such "exemplars" can be found by randomly choosing an initial subset of data points and then iteratively refining it, but this works well only if that initial choice is close to a good solution. We devised a method called "affinity propagation," which takes as input measures of similarity between pairs of data points. Real-valued messages are exchanged between data points until a high-quality set of exemplars and corresponding clusters gradually emerges. We used affinity propagation to cluster images of faces, detect genes in microarray data, identify representative sentences in this manuscript, and identify cities that are efficiently accessed by airline travel. Affinity propagation found clusters with much lower error than other methods, and it did so in less than one-hundredth the amount of time.C lustering data based on a measure of similarity is a critical step in scientific data analysis and in engineering systems. A common approach is to use data to learn a set of centers such that the sum of squared errors between data points and their nearest centers is small. When the centers are selected from actual data points, they are called "exemplars." The popular k-centers clustering technique (1) begins with an initial set of randomly selected exemplars and iteratively refines this set so as to decrease the sum of squared errors. k-centers clustering is quite sensitive to the initial selection of exemplars, so it is usually rerun many times with different initializations in an attempt to find a good solution. However, this works well only when the number of clusters is small and chances are good that at least one random initialization is close to a good solution. We take a quite different approach and introduce a method that simultaneously considers all data points as potential exemplars. By viewing each data point as a node in a network, we devised a method that recursively transmits real-valued messages along edges of the network until a good set of exemplars and corresponding clusters emerges. As described later, messages are updated on the basis of simple formulas that search for minima of an appropriately chosen energy function. At any point in time, the magnitude of each message reflects the current affinity that one data point has for choosing another data point as its exemplar, so we call our method "affinity propagation." Figure 1A illustrates how clusters gradually emerge during the message-passing procedure.

Approaches to working in high-dimensional data spaces: gene expression microarrays

“…However, unsupervised clustering constitutes an important tool for discovering underlying cancer subtypes or gene modules (Frey and Dueck, 2007;Miller et al, 2008). Such exploration may suggest possible refinement to established cancer categories, where cancer subtypes manifest radically different clinical behaviour and may correspond to distinct biological pathways involving subtype-specific markers (Shedden et al, 2003).…”

“…Rather than clustering samples using all genes, a practical alternative is to embed gene selection within unsupervised clustering -removal of noisy features improves clustering accuracy, which, in turn, guides a more accurate round of feature selection. Methods have been proposed along these lines (Xing and Karp, 2001;Graham and Miller, 2006), together with novel initialisation schemes (Frey and Dueck, 2007;Wang et al, 2007).…”

“…Specifically, such knowledge extraction should not strongly depend on the particular random sample of data used or on variable aspects of the algorithms. Many clustering algorithms find only locally optimal solutions whose quality depends on the pseudorandomly chosen initial cluster parameter values (Frey and Dueck, 2007). Also, greedy sequential feature selection techniques are often 'unstable', giving results that may be highly dependent upon the particular training data used.…”

A Novel Framework Using Graph-Based Clustering for Dental X-Ray Image Search in Medical Diagnosis

“…When k = i, point i is selected as an exemplar, or point k is the exemplar of point i. To reduce the oscillations when updating messages, damping factor  is introduced to iteration process (t is the iteration steps):AP have some main advantages such as: AP does not need to pre-assign the number of clusters; the greater value of preference, more number of clusters AP generates; AP also only accepts the collection of similarities as input, which eliminates the need to deal with the raw dataset directly; AP demonstrate its ability of processing large datasets rapidly and effectively from wide-ranging applications [10]. Update responsibility using equation (5) 4 Update availability using equation (6) 6 Until convergence…”

Feature-reduction and semi-simulated data in functional connectivity-based cortical parcellation

“…In this study, we proposed a feature-reduction approach based on the affinity propagation algorithm (APA) [28] and compared it with the classic approach based on principal component analysis (PCA) [29] . The approach based on APA provides feature-reduction by averaging the time courses of all voxels located within the same functional unit.…”