An Iterative Algorithm for Robust Kernel Principal Component Analysis

Abstract

Abstract:Principal Component Analysis (PCA) has been proven to be an efficient method in dimensionality reduction, feature extraction and pattern recognition. Kernel Principal Component Analysis (KPCA) can be considered as a natural nonlinear generalization of PCA, which performs linear PCA in a high dimensional space implicitly by using kernel trick. However, both conventional PCA and KPCA suffer from the deficiency of being sensitive to outliers. Existing robust KPCA has to eigen-decompose the Gram matrix directly in each step and is much more computationally infeasible due to the large size of the matrix when the number of training samples is large. By extending existing robust PCA algorithm using kernel methods, we present a novel robust adaptive algorithm for calculating the kernel principal components. The proposed method not only preserves the characteristic of capturing underlying nonlinear structure of KPCA but also is robust against outliers by restraining the effect of outlying samples. Compared with existing robust KPCA methods, our method is performed without having to store the kernel matrix, which can reduce significantly the storage burden. In addition, our method shows the potential of expansibility to the incremental learning version. Experimental results on synthetic data indicate that our improved algorithm is effective and promising.

Robust dimension reduction

“…In Ref a simple graphical procedure was proposed to detect and visualize the influential observations in ordinary Kernel PCA. Other related papers include Wang et al, Deng et al, Pang et al, and Huang and Yeh …”

“…In Ref 35 a simple graphical procedure was proposed to detect and visualize the influential observations in ordinary Kernel PCA. Other related papers include Wang et al, 39 Deng et al, 40 Pang et al, 41 and Huang and Yeh. 42 Another possible way to improve the robustness against outliers is by modifying the loss function.…”

LRSLibrary: Low-Rank and Sparse Tools for Background Modeling and Subtraction in Videos

“…Analysis on the exact recovery in the RPCA problem is given in [8]. In addition, extensive efforts have been made to analyze the RPCA problem from various perspectives [2,4,34,35,37]. In addition, extensive efforts have been made to analyze the RPCA problem from various perspectives [2,4,34,35,37].…”

A Variational Approach for Sparse Component Estimation and Low-Rank Matrix Recovery

“…Examples of algorithms in this category include the singular value thresholding (SVT) algorithm [11], the accelerated proximal gradient (APG) method [12], and the augmented Lagrange multiplier (ALM) method in [13]. In addition, extensive efforts have been made to analyze the RPCA problem from various perspectives [14]- [18].…”