Spectral Feature Scaling Method for Supervised Dimensionality Reduction

Abstract

Irregular features disrupt the desired classification. In this paper, we consider aggressively modifying scales of features in the original space according to the label information to form wellseparated clusters in low-dimensional space. The proposed method exploits spectral clustering to derive scaling factors that are used to modify the features. Specifically, we reformulate the Laplacian eigenproblem of the spectral clustering as an eigenproblem of a linear matrix pencil whose eigenvector has the scaling factors. Numerical experiments show that the proposed method outperforms well-established supervised dimensionality reduction methods for toy problems with more samples than features and real-world problems with more features than samples.

Projection method for eigenvalue problems of linear nonsquare matrix pencils

“…This study focuses on singular cases. This kind of problems (1.1) arises in the eigenstate computations of semiconductor quantum wells [1], the collocation method for approximating the eigenfunctions of the Hilbert-Schmidt operator [11], rectangular-shaped linear operators [4], and supervised dimensionality reduction [27,26]. Projection methods developed in [35] give the eigenvalues in a prescribed region and the corresponding eigenvectors of a regular matrix pencil.…”