Optimality and sub-optimality of PCA I: Spiked random matrix models

Abstract

A central problem of random matrix theory is to understand the eigenvalues of spiked random matrix models, introduced by Johnstone, in which a prominent eigenvector (or "spike") is planted into a random matrix. These distributions form natural statistical models for principal component analysis (PCA) problems throughout the sciences. Baik, Ben Arous and Péché showed that the spiked Wishart ensemble exhibits a sharp phase transition asymptotically: when the spike strength is above a critical threshold, it is possible to detect the presence of a spike based on the top eigenvalue, and below the threshold the top eigenvalue provides no information. Such results form the basis of our understanding of when PCA can detect a low-rank signal in the presence of noise. However, under structural assumptions on the spike, not all information is necessarily contained in the spectrum. We study the statistical limits of tests for the presence of a spike, including non-spectral tests. Our results leverage Le Cam's notion of contiguity, and include: i) For the Gaussian Wigner ensemble, we show that PCA achieves the optimal detection threshold for certain natural priors for the spike. ii) For any non-Gaussian Wigner ensemble, PCA is sub-optimal for detection. However, an efficient variant of PCA achieves the optimal threshold (for natural priors) by pre-transforming the matrix entries. iii) For the Gaussian Wishart ensemble, the PCA threshold is optimal for positive spikes (for natural priors) but this is not always the case for negative spikes. * The first two authors contributed equally.

Information-Theoretic Bounds and Phase Transitions in Clustering, Sparse PCA, and Submatrix Localization

“…Since the initial posting of this paper as an arXiv preprint, a number of interesting papers [44,43,35] have appeared, some extending or improving our results. Sharp lower bounds for sparse PCA were also obtained recently in [43] using a conditional second moment method similar to ours.…”

Message‐Passing Algorithms for Synchronization Problems over Compact Groups

“…The same analysis of the spectral method holds in this case; thus when > 1, the top eigenvector achieves nontrivial correlation with x, while for < 1, the spectral method fails and nontrivial estimation is provably impossible [54]. The same analysis of the spectral method holds in this case; thus when > 1, the top eigenvector achieves nontrivial correlation with x, while for < 1, the spectral method fails and nontrivial estimation is provably impossible [54].…”

“…This multifrequency Gaussian model was introduced by [54]. This multifrequency Gaussian model was introduced by [54].…”

“…A more elaborate Gaussian model is introduced in [54]: instead of observing only the matrix Y as above, suppose we are given matrices corresponding to different Fourier modes: A more elaborate Gaussian model is introduced in [54]: instead of observing only the matrix Y as above, suppose we are given matrices corresponding to different Fourier modes:…”

“…It is known that inefficient estimators can beat the D 1 threshold [54], but we conjecture that no efficient algorithm is able to break this barrier, thus concluding in Section 8 with an exploration of statisticalto-computational gaps that we expect to exist in synchronization problems, driven by ideas from statistical physics. We begin in Section 2 with an outline of our methods in the simplified cases of synchronization over Z=2 and U.1/, motivating our approach from a detailed discussion of prior work and its shortfalls.…”