A stepwise regression algorithm for high-dimensional variable selection

Abstract

We propose a new stepwise regression algorithm with a simple stopping rule for the identification of influential predictors and interactions among a huge number of variables in various statistical models. Like conventional stepwise regression, at each forward selection step, a variable is included in the current model if the test statistic of the enlarged model with the predictor against the current model has the minimum p-value among all the candidates and is smaller than a predetermined threshold. Instead of using conventional information types of criteria, the threshold is determined by a lower percentile of the beta distribution. We conducted extensive simulation studies to evaluate the performance of the proposed algorithm for various statistical models and found it to be very competitive and robust compared with several popular high-dimensional variable selection methods.

Consistent Estimation of Generalized Linear Models with High Dimensional Predictors via Stepwise Regression

“…Examples 1 and 3 were adopted from [ 7 ], while Examples 2 and 4 were borrowed from [ 5 , 15 ], respectively. We then generated the responses from the following three models.…”

“…To recap, our proposed method distinguishes from the existing stepwise approaches in high dimensional settings. For example, it improves [ 13 , 14 ] by extending the work to a more broad GLM setting and [ 15 ] by establishing the theoretical properties.…”

“…However, it is unclear whether the results hold for high-dimensional generalized linear models (GLMs); Ref. [ 15 ] proposed a similar stepwise algorithm in high-dimensional GLM settings, but with no theoretical properties on model selection. Moreover, none of the relevant works have touched upon the accuracy of estimation.…”

A new approach of dissecting genetic effects for complex traits

“…It is a fundamental problem in high-dimensional variable selection. Several methods have been developed to address these issues, such as LASSO 34,35 , stepwise regression [36][37][38] , penalized logistic regression 39 and penalized multiple regression 40 .…”

Efficient and accurate feature selection, with extensions for multiple solutions and to big data

“…A problem when using statistical tests for feature selection is that, due to multiple testing, the test statistics do not have the claimed distribution [69] and the resulting p-values are too small [56,67], leading to a high false discovery rate. Approaches to deal with problem include methods that dynamically adjusting significance levels [76], or methods that directly deal with the problem of sequential testing of stepwise by [158] and [154]. A method for dealing with model misspecification in model selection with information criterion is presented in [98].…”