Kernel Multivariate Analysis Framework for Supervised Subspace Learning: A Tutorial on Linear and Kernel Multivariate Methods

Abstract

Feature extraction and dimensionality reduction are important tasks in many fields of science dealing with signal processing and analysis. The relevance of these techniques is increasing as current sensory devices are developed with ever higher resolution, and problems involving multimodal data sources become more common. A plethora of feature extraction methods are available in the literature collectively grouped under the field of Multivariate Analysis (MVA). This paper provides a uniform treatment of several methods: Principal Component Analysis (PCA), Partial Least Squares (PLS), Canonical Correlation Analysis (CCA) and Orthonormalized PLS (OPLS), as well as their non-linear extensions derived by means of the theory of reproducing kernel Hilbert spaces. We also review their connections to other methods for classification and statistical dependence estimation, and introduce some recent developments to deal with the extreme cases of large-scale and low-sized problems. To illustrate the wide applicability of these methods in both classification and regression problems, we analyze their performance in a benchmark of publicly available data sets, and pay special attention to specific real applications involving audio processing for music genre prediction and hyperspectral satellite images for Earth and climate monitoring.

Emulation of Leaf, Canopy and Atmosphere Radiative Transfer Models for Fast Global Sensitivity Analysis

“…The scenario calls for the concept of regularization, which is tightly related to invariance encoding and incorporation of prior knowledge and the definition of sensible cost functions. Many opportunities appear here to improve the performance of emulators: one could think of including multiple pieces of information in the regression algorithm with multimodal/multiresolution regression, e.g., by combining RTMs for the same problem, to accommodate spatial or temporal relations in the emulation [44,94], and to implement better dimensionality reduction techniques beyond linear PCA to deal with the multi-output problem [95]. Apart from these improvements in the regression algorithm, we raise here the important issue of assessment of the emulator function, e.g., by looking at the Jacobian and Hessian of the transformation [38,96], Bayesian sensitivity analysis [34,97], as well as developing emulators that may deal with coupled RTMs and transformations of coefficients [50].…”

An Emulator Toolbox to Approximate Radiative Transfer Models with Statistical Learning

“…A common approach in statistics to alleviate these problems considers first reducing data dimensionality and then applying the OLS normal equations to the projected data or scores [38]. These scores reduce to a linear transformation of the original data, X = XU.…”

Kernel Supervised Ensemble Classifier for the Classification of Hyperspectral Data Using Few Labeled Samples

“…OPLS is a multivariate analysis method for feature extraction, which exploits the correlation between the features and the target data by combining the merits of canonical variate analysis and PLS [28,31,32]. Given a set of training samples {X,…”

“…OPLS is a variant of PLS, which is applicable to supervised problems, with certain optimality conditions regarding PLS. Moreover, considering that OPLS projections are obtained to predict the output labels, in consequence much more discriminative projection vectors are extracted compared to LDA, PLS [31,32].…”