LDA via L1-PCA of Whitened Data

Abstract

Principal component analysis (PCA) and Fisher's linear discriminant analysis (LDA) are widespread techniques in data analysis and pattern recognition. Recently, the L1-norm has been proposed as an alternative criterion to classical L2-norm in PCA, drawing considerable research interest on account of its increased robustness to outliers. The present work proves that, combined with a whitening preprocessing step, L1-PCA can perform LDA in an unsupervised manner, i.e., sparing the need for labelled data. Rigorous proof is given in the case of data drawn from a mixture of Gaussians. A number of numerical experiments on synthetic as well as real data confirm the theoretical findings.

Unsupervised and Computationally Lightweight Spectrum Sensing in IoT Devices

“…It can be shown that, under the whitening assumption, L1-PCA is useful in classification problems. Specifically, close connections with Fisher's linear discriminant analysis and the Fukunaga-Koontz transform (also known as 'common spatial patterns' technique) have been rigorously proven in [9,14]. Furthermore, L1-PCA can emulate these two techniques in a completely unsupervised manner.…”

“…These directions will be precisely those determined by L1-PCA, which, unlike the previous features, can be computed offline and stored on the device. This approach has its theoretical roots in the relationship between the L1 norm and the kurtosis [17], which is one of the classical features proposed by Nandi and Azzouz [16], and the discriminative characteristics of this variant of PCA [9,14]. For reasons of space, a more detailed explanation will be left for future papers.…”

Linear Convergence of a Proximal Alternating Minimization Method with Extrapolation for $\ell_1$-Norm Principal Component Analysis

“…Here, A 1 = i,j |A ij | denotes the ℓ 1 -norm of the matrix A. Besides being of interest in its own right, L1-PCA is also related to other data analytic tools, such as independent component analysis [29] and linear discriminant analysis [30]. However, unlike L2-PCA, which can essentially be solved in polynomial time, L1-PCA gives rise to a challenging computational problem.…”