Robust Subspace Learning: Robust PCA, Robust Subspace Tracking, and Robust Subspace Recovery

Abstract

PCA is one of the most widely used dimension reduction techniques. A related easier problem is "subspace learning" or "subspace estimation". Given relatively clean data, both are easily solved via singular value decomposition (SVD). The problem of subspace learning or PCA in the presence of outliers is called robust subspace learning or robust PCA (RPCA). For long data sequences, if one tries to use a single lower dimensional subspace to represent the data, the required subspace dimension may end up being quite large. For such data, a better model is to assume that it lies in a low-dimensional subspace that can change over time, albeit gradually. The problem of tracking such data (and the subspaces) while being robust to outliers is called robust subspace tracking (RST). This article provides a magazine-style overview of the entire field of robust subspace learning and tracking. In particular solutions for three problems are discussed in detail: RPCA via sparse+lowrank matrix decomposition (S+LR), RST via S+LR, and "robust subspace recovery (RSR)". RSR assumes that an entire data vector is either an outlier or an inlier. The S+LR formulation instead assumes that outliers occur on only a few data vector indices and hence are well modeled as sparse corruptions.

Enhancing the Spatio-Temporal Observability of Residential Loads

“…The recovery problem (7) is a convex problem that can be solved by generic convex optimization solvers. The computational complexity may grow fast with the matrix dimension, thereby calling for accelerated solutions such as alternating minimization [27] or adaptive updates using subspace learning approaches [28], which will be explored in the future to develop fast algorithms for solving (7). One main issue of the proposed recovery formulation (7) is that the L1-norm regularization may penalize the magnitude of nonzero entries of D. Hence, the solu-tionD tends to be smaller in magnitude than the actual values and is thus biased.…”

“…Certain conditions on K and D are required in order to achieve accurate RPCA results [9,28,29]. Loosely speaking, the low rank component K cannot be sparse and the sparse component D cannot be of low rank.…”

RES-PCA: A Scalable Approach to Recovering Low-Rank Matrices

“…The harder problem of seeking a PCA effective for outliercorruped data is called robust PCA (RPCA). There has been no mathematically precise meaning for the term "outlier" [24]. Thus multiple methods have been attempted to define or quantify this term, such as alternating minimization [14], random sampling techniques [9,17], multivariate trimming [11], and so on [7,27].…”

Deep Autoencoders with Value-at-Risk Thresholding for Unsupervised Anomaly Detection

“…The low-rank model corresponds to the background, and anomalies are outside of the low-rank subspace [8]- [12]. In Signal Processing [13]- [15], there is much related work on robust principal component analysis (RPCA) and subspace tracking. Extensive literature surveys can be found in [16]- [18].…”