Principal Polynomial Analysis

Abstract

This paper presents a new framework for manifold learning based on a sequence of principal polynomials that capture the possibly nonlinear nature of the data. The proposed Principal Polynomial Analysis (PPA) generalizes PCA by modeling the directions of maximal variance by means of curves, instead of straight lines. Contrarily to previous approaches, PPA reduces to performing simple univariate regressions, which makes it computationally feasible and robust. Moreover, PPA shows a number of interesting analytical properties. First, PPA is a volume-preserving map, which in turn guarantees the existence of the inverse. Second, such an inverse can be obtained in closed form. Invertibility is an important advantage over other learning methods, because it permits to understand the identified features in the input domain where the data has physical meaning. Moreover, it allows to evaluate the performance of dimensionality reduction in sensible (input-domain) units. Volume preservation also allows an easy computation of information theoretic quantities, such as the reduction in multi-information after the transform. Third, the analytical nature of PPA leads to a clear geometrical interpretation of the manifold: it allows the computation of Frenet-Serret frames (local features) and of generalized curvatures at any point of the space. And fourth, the analytical Jacobian allows the computation of the metric induced by the data, thus generalizing the Mahalanobis distance. These properties are demonstrated theoretically and illustrated experimentally. The performance of PPA is evaluated in dimensionality and redundancy reduction, in both synthetic and real datasets from the UCI repository.

Regression Wavelet Analysis for Progressive-Lossy-to-Lossless Coding of Remote-Sensing Data

“…Thus, despite the simplicity and efficiency that linearity provides, non-linear methods could provide performance improvements by better exploiting data dependencies. Recently, several non-linear generalization of PCA have been proposed to deal with data of non-linear nature [4,12,13]. In the adaptation of these non-linear methods to image coding, there are two major considerations: first, the transform must be invertible; and second, the computational complexity and the memory consumption should be reasonable.…”

“…In [14] we explored lossless hyperspectral image coding using curvilinear techniques based on Principal Polynomial Analysis (PPA) [13]. In that work, it was shown that PPA achieves higher energy compaction and statistical independence than PCA.…”

Regression Wavelet Analysis for Lossless Coding of Remote-Sensing Data

“…In [33], [34], the following families of nonlinear generalizations of feature extraction transforms were reviewed: (i) kernel and spectral techniques such as kernel-PCA, kernel-ICA or Local Linear Embedding (LLE) [35]- [37], (ii) neural networks and autoencoders [38]- [41], and (iii) techniques based on curvilinear features [10], [31], [33], [34], [42]- [44]. In the adaptation of these feature extraction ideas to image coding, there are two major considerations of importance: first, the transform must be invertible; and second, the computational complexity and the memory consumption should be reasonable.…”

“…In [46] we explored lossless hyperspectral image coding using curvilinear techniques (family iii) based on Principal Polynomial Analysis (PPA) [33]. PPA exploits regression to remove non-linear dependencies that remain after linear feature extraction (e.g., after classical PCA).…”

“…PPA exploits regression to remove non-linear dependencies that remain after linear feature extraction (e.g., after classical PCA). In [33] it was shown that PPA achieves higher energy compaction and statistical independence than PCA. However, in practical hyperspectral coding, appropriate handling of side information and the sequential error introduced by the integer mapping dramatically penalize the coding gain.…”