Building generalized linear models with ultrahigh dimensional features: A sequentially conditional approach

Abstract

Conditional screening approaches have emerged as a powerful alternative to the commonly used marginal screening, as they can identify marginally weak but conditionally important variables. However, most existing conditional screening methods need to fix the initial conditioning set, which may determine the ultimately selected variables. If the conditioning set is not properly chosen, the methods may produce false negatives and positives. Moreover, screening approaches typically need to involve tuning parameters and extra modeling steps in order to reach a final model. We propose a sequential conditioning approach by dynamically updating the conditioning set with an iterative selection process. We provide its theoretical properties under the framework of generalized linear models. Powered by an extended Bayesian information criterion as the stopping rule, the method will lead to a final model without the need to choose tuning parameters or threshold parameters. The practical utility of the proposed method is examined via extensive simulations and analysis of a real clinical study on predicting multiple myeloma patients’ response to treatment based on their genomic profiles.

Consistent Estimation of Generalized Linear Models with High Dimensional Predictors via Stepwise Regression

“…Overarching goals of ultra-high dimensional data analysis are to identify and estimate for . While most of the relevant literature [ 8 , 9 ] is on estimating , this work is to accomplish both identification of and estimation of .…”

“…Moreover, [ 8 ] proposed a sequentially conditioning approach based on offset terms that absorb the prior information. However, our numerical experiments indicate that the offset approach may be suboptimal compared to our full stepwise optimization approach, which will be demonstrated in the simulation studies.…”

“…Condition (1), as assumed in [ 8 , 18 ], is an alternative to the Lipschitz assumption [ 5 , 19 ]. The bound of the model size allowed in the selection procedure or q is often required in model-based screening methods see, e.g., [ 8 , 20 , 21 , 22 ]. The bound should be large enough so that the correct model can be included, but not too large; otherwise, excessive noise variables would be included, leading to unstable and inconsistent estimates.…”

“…Condition (5) represents the sparse Riesz condition [ 26 ] and Condition (6) is a strong "irrepresentable" condition, suggesting that cannot be represented by a set of variables that does not include the true model. It further implies that adding a signal variable to a mis-specified model will increase the log-likelihood by a certain lower bound [ 8 ]. The signal rate is comparable to the conditions required by the other sequential methods, see, e.g., [ 7 , 22 ].…”

“…For generating a final predictive model in ultrahigh dimensional settings, recent years have seen a surging interest of performing forward regression, an old technique for model selection; see [ 7 , 8 , 9 ], among many others. Under some regularity conditions and with some proper stopping criteria, forward regression can achieve screening consistency and sequentially select variables according to metrics such as AIC, BIC or .…”