A Locality Preserving Approach for Kernel PCA

Abstract

Dimensionality reduction is widely used in image understanding and machine learning tasks. Among these dimensionality reduction methods such as LLE, Isomap, etc., PCA is a powerful and efficient approach to obtain the linear low dimensional space embedded in the original high dimensional space. Furthermore, Kernel PCA (KPCA) is proposed to capture the nonlinear structure of the data in the projected space using "Kernel Trick ". However, KPCA fails to consider the locality preserving constraint which requires the neighboring points nearer in the reduced space. The locality constraint is natural and reasonable and thus can be incorporated into KPCA to improve the performance. In this paper, a novel method, which is called Locality Preserving Kernel PCA (LPKPCA) is proposed to reduce the reconstruction error and preserve the neighborhood relationship simultaneously. We formulate the objective function and solve it mathematically to derive the analytical solution. Several datasets have been used to compare the performance of KPCA and our novel LPKPCA including ORL face dataset, Yale Face Dataset B and Scene 15 Dataset. All the experimental results show that our method can achieve better performance on these datasets.