Phase Transitions for High Dimensional Clustering and Related Problems

Jin, Jiashun; Ke, Zheng Tracy; Wang, Wanjie

doi:10.48550/arxiv.1502.06952

Cited by 6 publications

(10 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Both versions of IF-PCA have better clustering results when δ = (1/3, 2/3), suggesting that the clustering task is more difficult in the symmetric case. This is consistent with the theoretical results; see for example Arias-Castro and Verzelen (2014); Jin, Ke and Wang (2015b). Experiment 2.…”

Section: Simulationssupporting

confidence: 91%

“…When (a)-(b) happen, it is almost impossible to successfully separate the useful features from useless ones, and it is preferable to use classical PCA. Such a scenario may be found in Jin, Ke and Wang (2015b); see for example Figure 1 (left) and related context therein.…”

mentioning

confidence: 84%

“…In a forthcoming manuscript Jin, Ke and Wang (2015b), we investigate the connections and differences between two problems more closely, and elaborate the above points with details.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Influential features PCA for high dimensional clustering

Jin¹,

Wang²

2016

Ann. Statist.

Self Cite

View full text Add to dashboard Cite

We consider a clustering problem where we observe feature vectors Xi ∈ R p , i = 1, 2, . . . , n, from K possible classes. The class labels are unknown and the main interest is to estimate them. We are primarily interested in the modern regime of p n, where classical clustering methods face challenges.We propose Influential Features PCA (IF-PCA) as a new clustering procedure. In IF-PCA, we select a small fraction of features with the largest Kolmogorov-Smirnov (KS) scores, obtain the first (K − 1) left singular vectors of the post-selection normalized data matrix, and then estimate the labels by applying the classical k-means procedure to these singular vectors. In this procedure, the only tuning parameter is the threshold in the feature selection step. We set the threshold in a data-driven fashion by adapting the recent notion of Higher Criticism. As a result, IF-PCA is a tuning-free clustering method.We apply IF-PCA to 10 gene microarray data sets. The method has competitive performance in clustering. Especially, in three of the data sets, the error rates of IF-PCA are only 29% or less of the error rates by other methods. We have also rediscovered a phenomenon on empirical null by Efron (2004) on microarray data.With delicate analysis, especially post-selection eigen-analysis, we derive tight probability bounds on the Kolmogorov-Smirnov statistics and show that IF-PCA yields clustering consistency in a broad context. The clustering problem is connected to the problems of sparse PCA and low-rank matrix recovery, but it is different in important ways. We reveal an interesting phase transition phenomenon associated with these problems and identify the range of interest for each. * Supported in part by NSF grants DMS-1208315 and DMS-1513414. MSC 2010 subject classifications: Primary 62H30, 62G32; secondary 62E20, 62P10.

show abstract

Section: Simulationssupporting

confidence: 91%

mentioning

confidence: 84%

See 1 more Smart Citation

Influential features PCA for high dimensional clustering

Jin¹,

Wang²

2016

Ann. Statist.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Let us consider the following two-class clustering problem in more detail (see Hastie et al (2009); Azizyan et al (2013); Jin and Wang (2016); Jin et al (2015)). Suppose l i ∈ {−1, 1}, i = 1, ..., n, are indicators representing the class label of the n-th nodes and let µ ∈ R p be a fixed vector.…”

Section: 3mentioning

confidence: 99%

“…Moreover, Jin and Wang (2016) and Jin et al (2015) considered the sparse and highly structured setting, where the contrast mean vector µ is assumed to be sparse and the nonzero coordinates are all equal. Their method is based on feature selection and PCA.…”

Section: 3mentioning

confidence: 99%

Rate-optimal perturbation bounds for singular subspaces with applications to high-dimensional statistics

Cai¹,

Zhang²

2018

Ann. Statist.

135

172

View full text Add to dashboard Cite

Perturbation bounds for singular spaces, in particular Wedin's sin Θ theorem, are a fundamental tool in many fields including highdimensional statistics, machine learning, and applied mathematics. In this paper, we establish separate perturbation bounds, measured in both spectral and Frobenius sin Θ distances, for the left and right singular subspaces. Lower bounds, which show that the individual perturbation bounds are rate-optimal, are also given.The new perturbation bounds are applicable to a wide range of problems. In this paper, we consider in detail applications to low-rank matrix denoising and singular space estimation, high-dimensional clustering, and canonical correlation analysis (CCA). In particular, separate matching upper and lower bounds are obtained for estimating the left and right singular spaces. To the best of our knowledge, this is the first result that gives different optimal rates for the left and right singular spaces under the same perturbation.

show abstract

A simple approach to sparse clustering

Arias-Castro

2017

Computational Statistics & Data Analysis

View full text Add to dashboard Cite

Consider the problem of sparse clustering, where it is assumed that only a subset of the features are useful for clustering purposes. In the framework of the COSA method of Friedman and Meulman, subsequently improved in the form of the Sparse K-means method of Witten and Tibshirani, a natural and simpler hill-climbing approach is introduced. The new method is shown to be competitive with these two methods and others.

show abstract

Phase Transitions for High Dimensional Clustering and Related Problems

Cited by 6 publications

References 31 publications

Influential features PCA for high dimensional clustering

Influential features PCA for high dimensional clustering

Rate-optimal perturbation bounds for singular subspaces with applications to high-dimensional statistics

A simple approach to sparse clustering

Contact Info

Product

Resources

About