Asymptotic performance of PCA for high-dimensional heteroscedastic data

Abstract

Principal Component Analysis (PCA) is a classical method for reducing the dimensionality of data by projecting them onto a subspace that captures most of their variation. Effective use of PCA in modern applications requires understanding its performance for data that are both high-dimensional and heteroscedastic. This paper analyzes the statistical performance of PCA in this setting, i.e., for high-dimensional data drawn from a low-dimensional subspace and degraded by heteroscedastic noise. We provide simplified expressions for the asymptotic PCA recovery of the underlying subspace, subspace amplitudes and subspace coefficients; the expressions enable both easy and efficient calculation and reasoning about the performance of PCA. We exploit the structure of these expressions to show that, for a fixed average noise variance, the asymptotic recovery of PCA for heteroscedastic data is always worse than that for homoscedastic data (i.e., for noise variances that are equal across samples). Hence, while average noise variance is often a practically convenient measure for the overall quality of data, it gives an overly optimistic estimate of the performance of PCA for heteroscedastic data.

Matrix Denoising for Weighted Loss Functions and Heterogeneous Signals

“…Remark 16. This noise model generalizes two previous models of heteroscedastic noise in the context of principal component analysis [21,22,23,55,39]. In both, the matrix X consists of random, i.i.d.…”

“…The model from [55,39] takes T = I n , in which case the observations are of the form Y j = X j + S 1/2 G j , where G j \sim N (0, I p ). By contrast, the papers [21,22,23] take S = I p , in which case the observations are of the form Y j =…”

Research on Credit Card Default Prediction Based on k-Means SMOTE and BP Neural Network

“…e main idea of the principal component analysis (PCA) method is to transform the n-dimensional feature variable through the coordinate axis and the origin to form a new m-dimensional feature (usually, m is less than n) [11].…”

Temporal scatterplots

“…Finally, a prominent alternative approach would be to facilitate Weighted-PCA (WPCA) [18]. In principle, WPCA was developed to address data samples with varying quality, by giving less informative samples less weight [34]. One could harness this approach by viewing the trajectory length as quality-the longer the length, the higher the weight of the points in the corresponding trajectory.…”