Extended BIC for small-n-large-P sparse GLM

Abstract

SummaryThe small-n-large-P situation has become common in genetics research, medical studies, risk management, and other fields. Feature selection is crucial in these studies yet poses a serious challenge. The traditional criteria such as AIC, BIC, and crossvalidation choose too many features. To overcome the difficulties caused by the small-n-large-P situation, Chen and Chen (2008) developed a family of extended Bayes information criteria (EBIC). Under normal linear models, EBIC is found to be consistent with nice finite sample properties. Proving consistency for non-normal and nonlinear models poses serious technical difficulties. In this paper, through a number of novel techniques, we establish the consistency of EBIC under generalized linear models in the small-n-large-P situation. We also report simulation results and a real-data analysis to illustrate the effectiveness of EBIC for feature selection.

Selection consistency of EBIC for GLIM with non-canonical links and diverging number of parameters

“…The selection consistency of EBIC for GLIMs with canonical links does not trivially pass to the case of non-canonical links. The selection consistency in the case of non-canonical links is established under more general conditions than those in Chen and Chen (2012). The conditions, though general, are naturally satisfied by many popular examples as given in Wedderburn (1976).…”

“…Conditions C2 and C3 are the same as conditions A2 and A3 in Chen and Chen (2012). Conditions C4-C5 reduce to conditions A4-A5 in Chen and Chen (2012) for canonical links.…”

“…Conditions C4-C5 reduce to conditions A4-A5 in Chen and Chen (2012) for canonical links. When A6 in Chen and Chen (2012) is satisfied, C6 is satisfied by commonly used GLIMs such as Poisson distribution with log and power function links, Binary distribution with identity, arcsin, complementary log-log and probit links, Gamma distribution with log and inverse power function links. These GLIMs are thoroughly studied in Wedderburn (1976).…”

“…The selection consistency of EBIC for linear models with a fixed number of parameters is established in Chen and Chen (2008). The result is then extended to generalized linear models (GLIM) with canonical links in Chen and Chen (2012). The EBIC has been used for choosing tuning parameters in penalized likelihood approaches, see Huang et al (2010), for feature selection procedures, see Wang (2009) and , and for QTL mapping and disease gene mapping studies, see Li and Chen (2009) and Zhao and Chen (2012).…”

Hypothesis Generation by Interactive Visual Exploration of Heterogeneous Medical Data

“…This results in datasets with small number of observations (small n) but a very high number of variables (large p). Since most of the statistical methods need sufficiently large number of observations to provide reliable estimates, such "long" data matrices lead to problematic computations [2]. Both the high dimensionality of the datasets and the "p ≫ n problem", pose big challenges for the analyst and the computational tools.…”