A robust variable screening procedure for ultra-high dimensional data

Abstract

Variable selection in ultra-high dimensional regression problems has become an important issue. In such situations, penalized regression models may face computational problems and some pre-screening of the variables may be necessary. A number of procedures for such pre-screening has been developed; among them the Sure Independence Screening (SIS) enjoys some popularity. However, SIS is vulnerable to outliers in the data, and in particular in small samples this may lead to faulty inference. In this paper, we develop a new robust screening procedure. We build on the density power divergence (DPD) estimation approach and introduce DPD-SIS and its extension iterative DPD-SIS. We illustrate the behavior of the methods through extensive simulation studies and show that they are superior to both the original SIS and other robust methods when there are outliers in the data. Finally, we illustrate its use in a study on regulation of lipid metabolism.

Robust Sure Independence Screening for Non-polynomial dimensional Generalized Linear Models

“…Since the existing theory of MDPDE (Basu et al, 2011;Ghosh and Basu, 2016) yields their increasing robustness for the increasing value of α > 0, the same is also expected for the proposed DPD-SIS with any α > 0. See also similar discussion in Ghosh and Thoresen (2020) for the linear regression model.…”

“…In this paper, we will utilize the MDPDEs under the marginal regression approach to develop a robust variable screening procedure for the ultra-high dimensional GLMs. Our proposal is thus an extension of the robust DPD-SIS of Ghosh and Thoresen (2020) to the class of GLMs, but we will additionally prove that the proposed procedure indeed satisfy the sure screening property and can also control the selection of false positive under appropriate assumptions on the covariates. The required assumptions are verified to hold under some mild conditions for common examples of GLMs.…”

“…Compared to any non-parametric robust procedure, a parametric robust approach is known to provide significantly better efficiency when the assumed model is valid for a majority of the data except only for the noise/contamination part (e.g., Hampel et al, 1986). Recently, a robust parametric version of SIS, namely the DPD-SIS, has been proposed for ultra-high dimensional linear regression models by Ghosh and Thoresen (2020). This DPD-SIS is empirically studied and found to have significantly improved performance compared to the other existing non-parametric SIS procedures under data contamination, although no theoretical guarantees are provided in Ghosh and Thoresen (2020).…”

“…Recently, a robust parametric version of SIS, namely the DPD-SIS, has been proposed for ultra-high dimensional linear regression models by Ghosh and Thoresen (2020). This DPD-SIS is empirically studied and found to have significantly improved performance compared to the other existing non-parametric SIS procedures under data contamination, although no theoretical guarantees are provided in Ghosh and Thoresen (2020). They have indeed proposed to use the marginal regression approach as in Fan and Song (2010) but to estimate the marginal regression slopes by the robust minimum density power divergence estimator (MDPDE) instead of the MLE.…”

“…Due to the high robustness of these MDPDEs along with high efficiency and simple computation, they are subsequently extended to more complex statistical models. For linear regression models, the MDPDEs are studied by Ghosh and Basu (2013) and subsequently used by Ghosh and Thoresen (2020). For different GLMs as well, the MDPDEs are seen to provide highly efficient and robust parameter estimates under the classical low dimensional set-ups (Basu et al, 2011;Ghosh and Basu, 2016;Basu et al, 2017Basu et al, , 2018Ghosh, 2019).…”