Hidden Integrality and Semirandom Robustness of SDP Relaxation for Sub-Gaussian Mixture Model

Abstract

We consider the problem of estimating the discrete clustering structures under the sub-Gaussian mixture model. Our main results establish a hidden integrality property of a semidefinite programming (SDP) relaxation for this problem: while the optimal solution to the SDP is not integer-valued in general, its estimation error can be upper bounded by that of an idealized integer program. The error of the integer program, and hence that of the SDP, are further shown to decay exponentially in the signal-to-noise ratio. In addition, we show that the SDP relaxation is robust under the semirandom setting in which an adversary can modify the data generated from the mixture model. In particular, we generalize the hidden integrality property to the semirandom model and thereby show that SDP achieves the optimal error bound in this setting. These results together highlight the “global-to-local” mechanism that drives the performance of the SDP relaxation. To the best of our knowledge, our result is the first exponentially decaying error bound for convex relaxations of mixture models. A corollary of our results shows that in certain regimes, the SDP solutions are in fact integral and exact. More generally, our results establish sufficient conditions for the SDP to correctly recover the cluster memberships of [Formula: see text] fraction of the points for any [Formula: see text]. As a special case, we show that under the [Formula: see text]-dimensional stochastic ball model, SDP achieves nontrivial (sometimes exact) recovery when the center separation is as small as [Formula: see text], which improves upon previous exact recovery results that require constant separation.

Clustering a Mixture of Gaussians with Unknown Covariance

“…Existing algorithms [71,24,85,75,34,46,23 Table 1: Comparison of sample complexity, misclassification rate, and computational complexity under different signal strength assumptions. Since SNR ≥ S, the bounds in the second column imply those in the third column.…”

“…In particular, when S 1 and n = Ω(d), it is known that Lloyd's algorithm [71,24], semi-definite relaxations of k-means [85,75,34,46,23] and spectral algorithms [1] achieve an error rate of e −Ω(S) . This rate depends suboptimally on SNR.…”

Clustering on Laplacian-embedded latent manifolds when clusters overlap

“…signed clustering [18]. Note further that the problem of recovering well separated clusters has been investigated through the lens of semidefinite programming [19][20][21][22][23], which gives an error bound on the recovered cluster matrix or even better proves exact recovery of the cluster labels (up to label permutations). It has also been addressed using non-negative matrix factorisation [24], which gives a probably approximately correct (PAC) Bayesian approach to the problem.…”