Sparse Convex Clustering

Abstract

Convex clustering, a convex relaxation of k-means clustering and hierarchical clustering, has drawn recent attentions since it nicely addresses the instability issue of traditional nonconvex clustering methods. Although its computational and statistical properties have been recently studied, the performance of convex clustering has not yet been investigated in the high-dimensional clustering scenario, where the data contains a large number of features and many of them carry no information about the clustering structure. In this paper, we demonstrate that the performance of convex clustering could be distorted when the uninformative features are included in the clustering. To overcome it, we introduce a new clustering method, referred to as Sparse Convex Clustering, to simultaneously cluster observations and conduct feature selection. The key idea is to formulate convex clustering in a form of regularization, with an adaptive group-lasso penalty term on cluster centers. In order to optimally balance the trade-off between the cluster fitting and sparsity, a tuning criterion based on clustering stability is developed. Theoretically, we obtain a finite sample error bound for our estimator and further establish its variable selection consistency. The effectiveness of the proposed method is examined through a variety of numerical experiments and a real data application.

Dynamic Visualization and Fast Computation for Convex Clustering via Algorithmic Regularization

“…Restricting our attention to standard convex clustering (p(·) = · q ), several useful methodological extensions have been proposed in the literature. For example, Wang et al (2016) augment the convex clustering problem (1) with an additional sparse component to add robustness to outliers, similar to the robust PCA formulation of Candès et al (2011), while Wang et al (2018) propose a variant which incorporates feature selection into the clustering objective using an 1 penalty (Tibshirani, 1996). As discussed in Section C, Chi et al The squared Frobenius loss function of the convex clustering problem may be interpreted as an isotropic Gaussian likelihood, suggesting another avenue for generalization.…”

Triangle Lasso for Simultaneous Clustering and Optimization in Graph Datasets

“…As a specific field of triangle lasso, convex clustering has drawn many attentions [2], [3], [5], [6], [7], [8], [9]. [5] proposes a new stochastic incremental algorithm to conduct convex clustering.…”

“…[2] uses an l 2,1 regularization to pick noisy features when conducting convex clustering. [3] investigates to remove sparse outlier or uninformative features when conducting convex clustering. However, both of them uses more than one convex regularized items in the formulation, which needs to tune multiple hyper-parameters in practical scenarios.…”

“…Real estate transactions (RET).The dataset is the real estate transactions over a week period in May 2008 in the Greater Sacramento area3 . The latitude and longitude features of each house are used to construct the graph.…”

The Guedon-Vershynin Semi-definite Programming Approach to Low Dimensional Embedding for Unsupervised Clustering

“…A nice method using ideas similar to the LASSO is ClusterPath [17]. This very interesting and efficient method has been studied and extended in [28], [26] and [32]. One of the main drawbacks of this approach is the lack of a robust rule for the choice of the parameters governing the procedure although they seem to be reasonably easy to tune in practice.…”