Two‐Stage Estimation for Ultrahigh Dimensional Sparse Quadratic Discriminant Analysis

Abstract

The conventional Quadratic Discriminant Analysis (QDA) encounters a significant hurdle due to parameter scaling complexities on the order of O(p2), rendering it impractical for the analysis of high or ultrahigh dimensional data. This arises especially when estimating the covariance matrix or its inverse, a necessity in such scenarios. In this research, we present an innovative two‐stage QDA procedure that mitigates this obstacle by reducing the dimensionality from p to a manageable level of o(min{n, p}). This reduction allows for a direct application of QDA even when the dimensionality is growing exponentially in terms of p. We observe that, under certain sparsity assumptions, the Bayes rule can be reformulated in a low‐dimensional form. This observation motivates us to select the most relevant classification features in the first stage using feature screening methods. Subsequently, we concentrate solely on this reduced subspace to formulate classifiers in the second stage. In addition to using QDA directly in the second stage, we introduce sparse QDA, resulting in three methods for constructing classifiers in the second stage. Under appropriate sparsity assumptions, we establish the consistency and misclassification rate of our proposed procedure. Numerical simulations and real data analyses demonstrate the effectiveness of our proposed method in finite‐sample scenarios.