Rapid Robust Principal Component Analysis: CUR Accelerated Inexact Low Rank Estimation

Cai, HanQin; Hamm, Keaton; Huang, Longxiu; Li, Jiaqi; Wang, Tao

doi:10.1109/lsp.2020.3044130

Cited by 36 publications

(22 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One of our future directions is to study learning-based stochastic approach. While this paper focuses on deterministic RPCA approaches, the stochastic RPCA approaches, e.g., partialGD [8], PG-RMC [9], and IRCUR [11], have shown promising speed advantage, especially for large-scale problems. Another future direction is to explore robust tensor decomposition incorporate with deep learning, as some preliminary studies have shown the advantages of tensor structure in certain machine learning tasks [45,46].…”

Section: Discussionmentioning

confidence: 99%

“…For the huge-scale problems where even a single truncated SVD is computationally prohibitive, one may replace the SVD step in initialization by some batch-based low-rank approximations, e.g., CUR decomposition. While its stability lacks theoretical support when outliers appear, the empirical evidence shows that a single CUR decomposition can provide sufficiently good initialization [11].…”

Section: Proposed Methodsmentioning

confidence: 99%

“…Over the last decade, robust principal component analysis (RPCA), one of the fundamental dimension reduction techniques, has received intensive investigations from theoretical and empirical perspectives [1][2][3][4][5][6][7][8][9][10][11][12][13]. RPCA also plays a key role in a wide range of machine learning tasks, such as video background subtraction [14], singing-voice separation [15], face modeling [16], image alignment [17] , feature identification [18], community detection [19], fault detection [20], and NMR spectrum recovery [21].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Learned Robust PCA: A Scalable Deep Unfolding Approach for High-Dimensional Outlier Detection

Cai

Liu

Yin

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Robust principal component analysis (RPCA) is a critical tool in modern machine learning, which detects outliers in the task of low-rank matrix reconstruction. In this paper, we propose a scalable and learnable non-convex approach for highdimensional RPCA problems, which we call Learned Robust PCA (LRPCA). LRPCA is highly efficient, and its free parameters can be effectively learned to optimize via deep unfolding. Moreover, we extend deep unfolding from finite iterations to infinite iterations via a novel feedforward-recurrent-mixed neural network model. We establish the recovery guarantee of LRPCA under mild assumptions for RPCA. Numerical experiments show that LRPCA outperforms the state-of-the-art RPCA algorithms, such as ScaledGD and AltProj, on both synthetic datasets and real-world applications.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Proposed Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learned Robust PCA: A Scalable Deep Unfolding Approach for High-Dimensional Outlier Detection

Cai

Liu

Yin

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…When the goal is to estimate the r leading eigenvalues and eigenvectors of the kernel matrix, the number of landmarks m should be greater than or equal to r. Recent empirical and theoretical results show that increasing the number of landmarks m and restricting the Nyström approximation to a lower rank-r space will lead to producing higher quality estimates [11], [15], [29], [38]. To explain the rank reduction procedure, we first compute the thin QR decomposition of the matrix C to find a factorization in the form of C = QR, where Q ∈ R n×m , Q T Q = I m , and R ∈ R m×m is an upper triangular matrix [39]. The second step is to compute the eigenvalue decomposition of the matrix B := RW † R T ∈ R m×m , which takes O(m 3 ) time and the number of landmarks m is assumed to be independent of n. Let B = VΣV T be the spectral decomposition of B, where the diagonal matrix Σ ∈ R m×m contains the eigenvalues of B in nonincreasing order and the columns of V ∈ R m×m are the eigenvectors.…”

Section: Preliminaries and Landmark Selection Techniquesmentioning

confidence: 99%

Kernel Matrix Approximation on Class-Imbalanced Data With an Application to Scientific Simulation

2021

View full text Add to dashboard Cite

Generating low-rank approximations of kernel matrices that arise in nonlinear machine learning techniques holds the potential to significantly alleviate the memory and computational burdens. A compelling approach centers on finding a concise set of exemplars or landmarks to reduce the number of similarity measure evaluations from quadratic to linear concerning the data size. However, a key challenge is to regulate tradeoffs between the quality of landmarks and resource consumption. Despite the volume of research in this area, current understanding is limited regarding the performance of landmark selection techniques in the presence of class-imbalanced data sets that are becoming increasingly prevalent in many applications. Hence, this paper provides a comprehensive empirical investigation using several realworld imbalanced data sets, including scientific data, by evaluating the quality of approximate low-rank decompositions and examining their influence on the accuracy of downstream tasks. Furthermore, we present a new landmark selection technique called Distance-based Importance Sampling and Clustering (DISC), in which the relative importance scores are computed for improving accuracy-efficiency tradeoffs compared to existing works that range from probabilistic sampling to clustering methods. The proposed landmark selection method follows a coarse-to-fine strategy to capture the intrinsic structure of complex data sets, allowing us to substantially reduce the computational complexity and memory footprint with minimal loss in accuracy.

show abstract

“…As the sparse coding and the summation all happen in the original domain, there is no additional transform-related computational burden per query in LSDAT. The most efficient computational complexity corresponding to RPCA is obtained by accelerated alternating projections algorithm (IRCUR) [10] which is (for…”

Section: Complexity and Convergence Rate Analysismentioning

confidence: 99%

LSDAT: Low-Rank and Sparse Decomposition for Decision-based Adversarial Attack

Esmaeili¹,

Edraki²,

Rahnavard³

et al. 2021

Preprint

View full text Add to dashboard Cite

We propose LSDAT, an image-agnostic decisionbased black-box attack that exploits low-rank and sparse decomposition (LSD) to dramatically reduce the number of queries and achieve superior fooling rates compared to the state-of-the-art decision-based methods under given imperceptibility constraints. LSDAT crafts perturbations in the low-dimensional subspace formed by the sparse component of the input sample and that of an adversarial sample to obtain query-efficiency. The specific perturbation of interest is obtained by traversing the path between the input and adversarial sparse components. It is set forth that the proposed sparse perturbation is the most aligned sparse perturbation with the shortest path from the input sample to the decision boundary for some initial adversarial sample (the best sparse approximation of shortest path, likely to fool the model). Theoretical analyses are provided to justify the functionality of LSDAT. Unlike other dimensionality reduction based techniques aimed at improving query efficiency (e.g, ones based on FFT), LSD works directly in the image pixel domain to guarantee that non-2 constraints, such as sparsity, are satisfied. LSD offers better control over the number of queries and provides computational efficiency as it performs sparse decomposition of the input and adversarial images only once to generate all queries. We demonstrate 0, 2 and ∞ bounded attacks with LSDAT to evince its efficiency compared to baseline decision-based attacks in diverse low-query budget scenarios as outlined in the experiments. 1

show abstract

Rapid Robust Principal Component Analysis: CUR Accelerated Inexact Low Rank Estimation

Cited by 36 publications

References 19 publications

Learned Robust PCA: A Scalable Deep Unfolding Approach for High-Dimensional Outlier Detection

Learned Robust PCA: A Scalable Deep Unfolding Approach for High-Dimensional Outlier Detection

Kernel Matrix Approximation on Class-Imbalanced Data With an Application to Scientific Simulation

LSDAT: Low-Rank and Sparse Decomposition for Decision-based Adversarial Attack

Contact Info

Product

Resources

About