Nonlinear Component Analysis as a Kernel Eigenvalue Problem

Schölkopf, Bernhard; Smola, Alexander J.; Müller, Klaus‐Robert

doi:10.1162/089976698300017467

Cited by 6,836 publications

(3,458 citation statements)

References 16 publications

Supporting

Mentioning

3,413

Contrasting

Unclassified

Order By: Relevance

“…Kernel Principal Component Analysis (kernel PCA) (Scholkopf et al, 1998) has been proposed as a nonlinear extension of the standard PCA and has been applied to various purposes including feature extraction, denoising and pre-processing of regression. Kernel PCA is an example of the so-called kernel methods (Scholkopf and Smola, 2002), which aim to extract nonlinear features of the original data by mapping them into a high-dimensional feature space Reproducing Kernel Hilbert Space (RKHS).…”

Section: Introductionmentioning

confidence: 99%

Hyperparameter Selection in Kernel Principal Component Analysis

Alam

Fukumizu²

2014

Journal of Computer Science

View full text Add to dashboard Cite

In kernel methods, choosing a suitable kernel is indispensable for favorable results. No well-founded methods, however, have been established in general for unsupervised learning. We focus on kernel Principal Component Analysis (kernel PCA), which is a nonlinear extension of principal component analysis and has been used electively for extracting nonlinear features and reducing dimensionality. As a kernel method, kernel PCA also suffers from the problem of kernel choice. Although cross-validation is a popular method for choosing hyperparameters, it is not applicable straightforwardly to choose a kernel in kernel PCA because of the incomparable norms given by different kernels. It is important, thus, to develop a wellfounded method for choosing a kernel in kernel PCA. This study proposes a method for choosing hyperparameters in kernel PCA (kernel and the number of components) based on cross-validation for the comparable reconstruction errors of pre-images in the original space. The experimental results on synthesized and real-world datasets demonstrate that the proposed method successfully selects an appropriate kernel and the number of components in kernel PCA in terms of visualization and classification errors on the principal components. The results imply that the proposed method enables automatic design of hyperparameters in kernel PCA.

show abstract

Section: Introductionmentioning

confidence: 99%

Hyperparameter Selection in Kernel Principal Component Analysis

Alam

Fukumizu²

2014

Journal of Computer Science

View full text Add to dashboard Cite

show abstract

“…We can view kPCA as applying a nonlinear transformation to the m-dimensional features x, φ(x) ∈ R q , to obtain a higher dimensional representation q m. [37,38] However, this might be computationally expensive, especially if the number of dimensions in the original data is high. We therefore use the so-called kernel function to obtain the kernel matrix of the data K(x i , x j ), i, j = 1, · · · , N .…”

Section: Kernel-principal Component Analysismentioning

confidence: 99%

Assessment of feature selection and classification methods for recognizing motor imagery tasks from electroencephalographic signals

Vega

Sajed

Mathewson

et al. 2016

AIR

View full text Add to dashboard Cite

Recognition of motor imagery tasks (MI) from electroencephalographic (EEG) signals is crucial for developing rehabilitation and motor assisted devices based on brain-computer interfaces (BCI). Here we consider the challenge of learning a classifier, based on relevant patterns of the EEG signals; this learning step typically involves both feature selection, as well as a base learning algorithm. However, in many cases it is not clear what combination of these methods will yield the best classifier. This paper contributes a detailed assessment of feature selection techniques, viz., squared Pearson's correlation (R 2 ), principal component analysis (PCA), kernel principal component analysis (kPCA) and fast correlation-based filter (FCBF); and the learning algorithms: linear discriminant analysis (LDA), support vector machines (SVM), and Feed Forward Neural Network (NN). A systematic evaluation of the combinations of these methods was performed in three two-class classification scenarios: rest vs. movement, upper vs. lower limb movement and right vs. left hand movement. FCBF in combination with SVM achieved the best results with a classification accuracy of 81.45%, 77.23% and 68.71% in the three scenarios, respectively. Importantly, FCBF determines, based on the feature set, whether a classifier can be learned, and if so, automatically identifies the subset of relevant and non-correlated features. This suggests that FCBF is a powerful method for BCI systems based on MI. Knowledge gained here about procedural combinations has the potential to produce useful BCI tool, that can provide effective motor control for the users.

show abstract

“…, we use the kernel principal component analysis (kPCA) method established by Schölkopf et al in [35]. They showed that λ k and {α k i } m i=1 can be found in terms of the eigenvalues and eigenvectors of a centered kernel matrix.…”

Section: Shape Prior Regularization Termmentioning

confidence: 99%

“…when using (35) to obtain the last equality. Comparing (36) and (30) it is clear that the problem of calculating J prior has been reduced to finding a suitable centered kernel functionk.…”

Section: Shape Prior Regularization Termmentioning

confidence: 99%