A fast incremental Kernel Principal Component Analysis for learning stream of data chunks

Abstract

In this paper, a new incremental learning algo rithm of Kernel Principal Component Analysis (KPCA) is proposed for online feature extraction in pattern recognition problems. The proposed algorithm is derived by extending the Takeuchi et al.'s Incremental KPCA (T-IKPCA) that can learn a new data incrementally without keeping past training data.However, even if more than two data are given in a chunk, T-IKPCA should learn them individually; that is, in order to update the eigen-feature space, the eigenvalue decomposition should be performed for every data in the chunk. To alleviate this problem, we extend T-IKPCA such that an eigen-feature space learning is conducted by performing the eigenvalue decomposition only once for a chunk of given data. In the proposed IKPCA, whenever a new chunk of training data are given, linearly independent data are first selected based on the cumulative proportion. Then, the eigenspace augmentation is conducted by calculating the coefficients for the selected linearly independent data, and the eigen-feature space is rotated based on the rotation matrix that can be obtained by solving a kernel eigenvalue problem. To verify the effectiveness of the proposed IKPCA, the learning time and the accuracy of eigenvectors are evaluated using the three VCI benchmark data sets. From the experimental results, we confirm that the proposed IKPCA can learn an eigen-feature space very fast without sacrificing the recognition accuracy.

Incremental kernel PCA and the Nyström method

“…Various approximations to incremental kernel PCA have also been proposed. See for example Tokumoto and Ozawa (2011) or Sheikholeslami et al (2015). Since we present an exact algorithm for incremental kernel PCA, we will not describe these or similar works further.…”

A property of learning chunk data using incremental kernel principal component analysis

“…where the threshold (J is determined using the k-fold cross validation method (see [10] for details). as the ( m + l ) th independent data 4>(xm+d and added to �m [10]:…”

“…Recently, Tokumoto and Ozawa [10] have proposed a fast version of Takeuchi et al's IKPCA [9] called Chunk IKPCA (CIPCA) which can reduce the number of times to perform the eigenvalue decomposition when a small number of data are simultaneously given as a stream of data chunks. In CIKPCA, linearly independent data are first selected from a chunk of new data to augment eigenvectors and some of them are kept as an independent data set until the cumulative proportion becomes larger than a threshold.…”