Compressive-Projection Principal Component Analysis

Abstract

Principal component analysis (PCA) is often central to dimensionality reduction and compression in many applications, yet its data-dependent nature as a transform computed via expensive eigendecomposition often hinders its use in severely resource-constrained settings such as satellite-borne sensors. A process is presented that effectively shifts the computational burden of PCA from the resource-constrained encoder to a presumably more capable base-station decoder. The proposed approach, compressive-projection PCA (CPPCA), is driven by projections at the sensor onto lower-dimensional subspaces chosen at random, while the CPPCA decoder, given only these random projections, recovers not only the coefficients associated with the PCA transform, but also an approximation to the PCA transform basis itself. An analysis is presented that extends existing Rayleigh-Ritz theory to the special case of highly eccentric distributions; this analysis in turn motivates a reconstruction process at the CPPCA decoder that consists of a novel eigenvector reconstruction based on a convex-set optimization driven by Ritz vectors within the projected subspaces. As such, CPPCA constitutes a fundamental departure from traditional PCA in that it permits its excellent dimensionality-reduction and compression performance to be realized in an light-encoder/heavy-decoder system architecture. In experimental results, CPPCA outperforms a multiple-vector variant of compressed sensing for the reconstruction of hyperspectral data.

Sparse Sensor Placement Optimization for Classification

“…Sparse approximation applied to semantic hierarchies [41] has been shown to be efficient in categorizing between large numbers of classes [35]. Ideas from compressed sensing have also been applied to dimensionality reduction tools to develop a theory of sketched SVD based on randomly projected data [23,47,29]. Methods to promote sparsity and parsimonious representations of high-dimensional data from few measurements include the development of sparse PCA (SPCA) [64] and, in a related line of work in feature selection for classification, penalized and sparse LDA [60,16].…”

“…Therefore, we also develop an intermediate technique, related to the sketched SVD [23,28,29], to start instead with a subsample of the original data. We demonstrate that starting with 10% of the original data, we are still able to find nearly optimal sparse sensor locations.…”

Decoder-side dimensionality determination for compressive-projection principal component analysis of hyperspectral data

“…We now briefly overview the CPPCA procedure; the reader is referred to [1] for a more complete description. Consider…”

“…Specifically, in the CPPCA decoder, the first L principal eigenvectors of the PCA transform basis are approximately recovered from the random projections of the data. An open problem not discussed in [1] is how the CPPCA decoder determines how many principal eigenvectors it should attempt to recover from the random projections.…”

“…This heavy computation burden hinders the deployment of PCA in resource-constrained environments such as satellite-borne sensor platforms. In order to make PCA more amenable to such remote devices, [1] developed a process to shift the computational burden from a resource-constrained sensor to a more powerful base-station decoder. This process, called compressive-projection PCA (CPPCA), is driven by projections at the sensor onto a randomly-chosen lower-dimensional subspace.…”