Variable Selection via Partial Correlation

Abstract

Partial correlation based variable selection method was proposed for normal linear regression models by Bühlmann, Kalisch and Maathuis (2010) as a comparable alternative method to regularization methods for variable selection. This paper addresses two important issues related to partial correlation based variable selection method: (a) whether this method is sensitive to normality assumption, and (b) whether this method is valid when the dimension of predictor increases in an exponential rate of the sample size. To address issue (a), we systematically study this method for elliptical linear regression models. Our finding indicates that the original proposal may lead to inferior performance when the marginal kurtosis of predictor is not close to that of normal distribution. Our simulation results further confirm this finding. To ensure the superior performance of partial correlation based variable selection procedure, we propose a thresholded partial correlation (TPC) approach to select significant variables in linear regression models. We establish the selection consistency of the TPC in the presence of ultrahigh dimensional predictors. Since the TPC procedure includes the original proposal as a special case, our theoretical results address the issue (b) directly. As a by-product, the sure screening property of the first step of TPC was obtained. The numerical examples also illustrate that the TPC is competitively comparable to the commonly-used regularization methods for variable selection.

Variable screening with multiple studies

“…The second condition (C2) allows the dimension p to grow at an exponential rate of sample size n, which is a fairly standard assumption in high-dimensional analysis. Many sure screening methods like "SIS", "DC-SIS" and "TPC" have used this assumption (Fan and Lv, 2008;Li et al, 2012Li et al, , 2017. Though the PC-simple algorithm (Bühlmann et al, 2010) assumes a polynomial growth of p n as a function of n, we notice that it can be readily relaxed to an exponential of n level.…”

Variable selection for partially linear models via partial correlation

“…For the sake of simplicity, the least squared estimates trueβ^js are computed for the nonzero coefficients, and the nonparametric function is estimated by g^(u)=S(y−Xboldβ^) with the plug-in trueβ^. We summarize the whole procedure in Algorithm 1, in which we follow [10] to set T(α,n,κ,|scriptS|)=expfalse{21+κΦ−1(1−α/2)/n−|scriptS|−1false}−1expfalse{21+κΦ−1(1−α/2)/n−|scriptS|−1false}+1 and κ^=1ptrue∑j=1p[n−1…”

“…In this paper, we propose a new variable selection procedure for PLM. This procedure differs from the aforementioned penalized least squares methods in that it is a partial correlation learning procedure based on the notion of partial faithfulness that was first advocated by Buhlmann et al [1] for normal linear models and further used for elliptical linear models in [10]. We first utilize partial residual techniques to eliminate the nonparametric baseline function, and then conduct variable selection by recursively testing the partial correlations between the partial residual of the response and that of the linear covariates.…”

“…That is, we recursively compare the partial correlations with some threshold values, and therefore refer to this method as the thresholded partial correlation on partial residuals (TPC-PR). Thus, the TPC-PR can be carried out by using the algorithm proposed in [10].…”

A Two-Stage Mutual Information Based Bayesian Lasso Algorithm for Multi-Locus Genome-Wide Association Studies

“…The extension of SIS is iterative sure independence screening (ISIS), which can revive those non-negligible features that are single uncorrelated while indirectly correlated to the respond variables [ 11 ]. Instead of the Pearson correlation based SIS, statisticians have exploited some other SIS methods from different measurements, such as rank correlation [ 14 ], the distance correlation [ 15 ], the partial correlation [ 16 ] and so on. Among these methods, Pearson correlation and distance correlation based screening have been applied in GWAS successfully [ 6 , 17 ], and some genes associated with crop quantitative traits such as rice salt-tolerance and poplar growth have been identified [ 18 , 19 ].…”