A subspace recursive and selective feature transformation method for classification tasks

Abstract

Background: Practitioners and researchers often found the intrinsic representations of high-dimensional problems has much fewer independent variables. However such intrinsic structure may not be easily discovered due to noises and other factors. A supervised transformation scheme RST is proposed to transform features into lower dimensional spaces for classification tasks. The proposed algorithm recursively and selectively transforms the features guided by the output variables. Results: We compared the classification performance of linear classifier and random forest classifier on the original data sets, data sets being transformed with RST and data sets being transformed by principle component analysis and linear discriminant analysis. On 7 out 8 data sets RST shows superior classification performance with linear classifiers but less ideal with random forest classifiers. Conclusions: Our test shows the proposed method's capability to reduce features dimensions in general classification tasks and preserve useful information using linear transformations. Some limitations of this method are also pointed out.