Optimal classification in sparse Gaussian graphic model

Abstract

Consider a two-class classification problem where the number of features is much larger than the sample size. The features are masked by Gaussian noise with mean zero and covariance matrix $\Sigma$, where the precision matrix $\Omega=\Sigma^{-1}$ is unknown but is presumably sparse. The useful features, also unknown, are sparse and each contributes weakly (i.e., rare and weak) to the classification decision. By obtaining a reasonably good estimate of $\Omega$, we formulate the setting as a linear regression model. We propose a two-stage classification method where we first select features by the method of Innovated Thresholding (IT), and then use the retained features and Fisher's LDA for classification. In this approach, a crucial problem is how to set the threshold of IT. We approach this problem by adapting the recent innovation of Higher Criticism Thresholding (HCT). We find that when useful features are rare and weak, the limiting behavior of HCT is essentially just as good as the limiting behavior of ideal threshold, the threshold one would choose if the underlying distribution of the signals is known (if only). Somewhat surprisingly, when $\Omega$ is sufficiently sparse, its off-diagonal coordinates usually do not have a major influence over the classification decision. Compared to recent work in the case where $\Omega$ is the identity matrix [Proc. Natl. Acad. Sci. USA 105 (2008) 14790-14795; Philos. Trans. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci. 367 (2009) 4449-4470], the current setting is much more general, which needs a new approach and much more sophisticated analysis. One key component of the analysis is the intimate relationship between HCT and Fisher's separation. Another key component is the tight large-deviation bounds for empirical processes for data with unconventional correlation structures, where graph theory on vertex coloring plays an important role.Comment: Published in at http://dx.doi.org/10.1214/13-AOS1163 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

On set-based association tests: insights from a regression using summary statistics

“…It is worth emphasizing three notable differences between the proposed T ME and the well-known T skato (Lee, Wu & Lin, 2012). First, the proposed framework aggregates evidence across J variants based on , a precision matrix-based transformation of the score vector, that can increase power for detecting sparse alternatives (Fan, Jin & Yao, 2013; Cai, Liu & Xia, 2014). Secondly, unlike T skato , T ME naturally aggregates the information from the linear and quadratic types of tests, based on a regression model without searching for the ‘optimal’ weight.…”

“…To this end, we propose a flexible and unifying linear mixed-effect regression model that requires only variant-specific summary statistics, and we show that earlier methods based on individual-level data are special cases of the proposed testing framework. The set-based association test statistic derived from the regression model inherently transforms the variant-specific summary statistics using the precision matrix to improve power for detecting sparse alternatives; see Fan, Jin & Yao (2013) and Cai, Liu & Xia (2014) for the use of precision matrix in other high-dimension analytical settings. Furthermore, the proposed method can incorporate additional variant-specific information as a covariate, e.g.…”

Integrative Genetic Risk Prediction Using Non-Parametric Empirical Bayes Classification

“…A popular alternative is to use some form of regularized estimation. A number of strategies for high‐dimensional discriminant analysis have been developed in this vein, though mostly under the assumption that the are normal rather than binomial random variables (Fan et al., ; Greenshtein and Park, ; Cai and Liu, ; Fan et al., ; Mai et al., ; Fan et al., ; Han et al., ; Dicker and Zhao, ).…”

Dynamic linear discriminant analysis in high dimensional space

“…Fortunately, in many problems, various quantities in the LDA can be assumed sparse; See, for example, Witten & Tibshirani (2011);Shao et al (2011);Cai & Liu (2011);Fan, Feng & Tong (2012); Mai, Zou & Yuan (2012), and Mai & Zou (2013) for a summary of selected sparse LDA methods. Further studies along this line can be found in Fan, Jin & Yao (2013) and Hao, Dong & Fan (2015). More recently, quadratic discriminant analysis has attracted increasing attention where the population covariance matrices are assumed static but different.…”