Innovated interaction screening for high-dimensional nonlinear classification

Abstract

This paper is concerned with the problems of interaction screening and nonlinear classification in a high-dimensional setting. We propose a two-step procedure, IIS-SQDA, where in the first step an innovated interaction screening (IIS) approach based on transforming the original p-dimensional feature vector is proposed, and in the second step a sparse quadratic discriminant analysis (SQDA) is proposed for further selecting important interactions and main effects and simultaneously conducting classification. Our IIS approach screens important interactions by examining only p features instead of all two-way interactions of order O(p 2 ). Our theory shows that the proposed method enjoys sure screening property in interaction selection in the high-dimensional setting of p growing exponentially with the sample size. In the selection and classification step, we establish a sparse inequality on the estimated coefficient vector for QDA and prove that the classification error of our procedure can be upperbounded by the oracle classification error plus some smaller order term. Extensive simulation studies and real data analysis show that our proposal compares favorably with existing methods in interaction selection and high-dimensional classification.

Classification with high dimensional features

“…QUADRO proposed by Fan, Ke, Liu, and Xia (2015) is a high-dimensional generalization of quadratic discriminant analysis. Fan, Kong, Li, and Zheng (2015) proposed a two-step procedure called IIS-SQDA for high-dimensional classification under a QDA model where a new feature/interaction screen technique is used to reduce the dimensionality.…”

An integrated inverse space sparse representation framework for tumor classification

“…In addition to these classical classifiers, our method has also compared with the latest published classification results on the same breast dataset [48][49][54][55][56][57], and some other state-of-the-art SRC methods, including PFSRC [14], RRC_L1 [43] and RRC_L2 [43]. Table 11 shows that, on the Breast-2 (77) Breast-2(97), classification accuracy of our method higher than those of in the latest published results given in the same dataset and same environment.…”

Intentional Control of Type I Error Over Unconscious Data Distortion: A Neyman–Pearson Approach to Text Classification

“…In the statistics literature, various methods have been proposed to reduce the feature dimensionality. For example, one can use marginal screening methods such as sure independence screening (Fan and Lv, 2008), nonparametric independence screening (Fan et al, 2011) and Kolmogorov-Smirnov (KS) test, interaction screening methods (Hao and Zhang, 2014;Fan et al, 2015), the forward stepwise selection, shrinkage methods such as LASSO (Tibshirani, 1996) and SCAD (Fan and Li, 2001), or dimension reduction methods such as principal component analysis.…”