Model-Free Feature Screening for Ultrahigh Dimensional Data through a Modified Blum-Kiefer-Rosenblatt Correlation

Abstract

In this paper we introduce a modified Blum-Kiefer-Rosenblatt correlation (MBKR for short) to rank the relative importance of each predictor in ultrahigh dimensional regressions. We advocate using the MBKR for two reasons. First, the MBKR is nonnegative and equals zero if and only if two random variables are independent, indicating that the MBKR can detect nonlinear dependence. We illustrate that the sure independence screening procedure based on the MBKR (MBKR-SIS for short) is effective to detect nonlinear effects including interactions and heterogeneity, particularly when both continuous and discrete predictors are involved simultaneously. Second, the MBKR is conceptually simple, easy to implement and affine-invariant. The MBKR is free of tuning parameters and no iteration is required in estimation. It remains unchanged when order-preserving transformations are applied to the response or predictors, indicating that the MBKR-SIS is robust to the presence of extreme values and outliers in the observations. We also show that, under mild conditions, the MBKR-SIS procedure has the desirable sure screening and ranking consistency properties, Statistica Sinica: Newly accepted Paper (accepted version subject to English editing) Feature Screening via Modified Blum-Kiefer-Rosenblatt Correlation 2 which guarantee that all important predictors can be retained after screening with probability approaching one. We also propose an iterative screening procedure to detect the important predictors which are marginally independent of the response variable. We demonstrate the merits of the MBKR-SIS procedure through simulations and an application to a real-world dataset.

Composite Coefficient of Determination and Its Application in Ultrahigh Dimensional Variable Screening

“…Proof of Theorem 3. The proof of Theorem 3 shares the similar spirit of Theorem 2 in Fan, Feng and Song (2011) and Theorem 3 in Zhou and Zhu (2018).…”

Error‐controlled feature selection for ultrahigh‐dimensional and highly correlated feature space using deep learning

“…One limitation is that we mainly focus on the regression setup with the continuous outcome because of the requirements of the HZ sure Independence test used in the screening step. For a classification task, any model-free feature screening method like Zhou & Zhu [54] can be applied in a more general framework.…”

High‐dimensional model averaging for quantile regression

“…To generate more reliable results, Scheetz et al applied the expression quantitative trait locus (eQTL) mapping method to 18,976 probes that are considered as sufficient variables and exhibit at least two‐fold variation in expression level. Following Huang et al (2010) and Zhou & Zhu (2018), our goal is to identify genes correlated with gene TRIM32, which causes the Bardet–Biedl syndrome. The probe from TRIM32 is 1389163at, which is one of the 18,976 probes that are sufficiently expressed and can be regarded as response .…”