Diffusion K-means clustering on manifolds: Provable exact recovery via semidefinite relaxations

Abstract

We introduce the diffusion K-means clustering method on Riemannian submanifolds, which maximizes the within-cluster connectedness based on the diffusion distance. The diffusion K-means constructs a random walk on the similarity graph with vertices as data points randomly sampled on the manifolds and edges as similarities given by a kernel that captures the local geometry of manifolds. Thus the diffusion K-means is a multi-scale clustering tool that is suitable for data with non-linear and non-Euclidean geometric features in mixed dimensions. Given the number of clusters, we propose a polynomial-time convex relaxation algorithm via the semidefinite programming (SDP) to solve the diffusion K-means. In addition, we also propose a nuclear norm (i.e., trace norm) regularized SDP that is adaptive to the number of clusters. In both cases, we show that exact recovery of the SDPs for diffusion K-means can be achieved under suitable between-cluster separability and within-cluster connectedness of the submanifolds, which together quantify the hardness of the manifold clustering problem. We further propose the localized diffusion K-means by using the local adaptive bandwidth estimated from the nearest neighbors. We show that exact recovery of the localized diffusion K-means is fully adaptive to the local probability density and geometric structures of the underlying submanifolds.

Forest Fire Clustering for single-cell sequencing combines iterative label propagation with parallelized Monte Carlo simulations

“…Many existing methods mimic diffusion or use Monte Carlo for information processing 35 , 36 . Indeed, the label propagation in Forest Fire Clustering represents a simplified version of diffusion and is similar to region-growing algorithms for image processing.…”

Understanding and design of non-conservative optical matter systems using Markov state models

“…88 Chen and Yang introduced diffusion k -means, which maximizes the within-cluster connectedness based on the diffusion distance. 61 The diffusion distance is defined as the Euclidean distance in the eigenvector space of diffusion map embedding. 60,72,73 In other words, diffusion k -means is k -means clustering applied to diffusion map embeddings.…”

“…Our analysis uses a permutation invariant pairwise metric that we supply as the kernel to perform nonlinear dimensionality reduction using diffusion maps. 60 We use diffusion k -means 61 to define the microstate clustering, and robust Perron cluster cluster analysis (PCCA+) 62–66 to define the macrostate clustering and build MSMs. We test the Markovianity of the macrostate MSMs using the Chapman–Kolmogorov (CK) test to verify that they are valid kinetic models of the non-equilibrium OM system and provide post hoc support for the use of permutationally-invariant diffusion map embeddings to identify and resolve microstates.…”

Sketch-and-Lift: Scalable Subsampled Semidefinite Program for $K$-means Clustering

“…Fast approximation algorithms to solve the K-means such as Lloyd's algorithm [19,20] and spectral clustering [22,26,31,2,32,33] provably yield consistent recovery when different groups are well separated. Recently, semi-definite programming (SDP) relaxations [27,23,18,13,11,29,14,9] have emerged as an important approach for clustering due to its superior empirical performance [27], robustness against outliers and adversarial attack [13], and attainment of the information-theoretic limit [10]. Despite having polynomial time complexity, the SDP relaxed Kmeans has notoriously poor scalability to large (or even moderate) datasets for instance by interior point methods [3,15].…”