The Conditional Permutation Test for Independence While Controlling for Confounders

Berrett, Thomas B.; Wang, Yi; Barber, Rina Foygel; Samworth, Richard J.

doi:10.1111/rssb.12340

Cited by 87 publications

(82 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Constructing nontrivial knockoffs is a considerable challenge. Numerous methods have been proposed, including but not limited to: second-order Gaussian knockoffs (Candès et al, 2018); conditional permutation sampling (Berrett et al, 2020); hidden Markov models (Sesia et al, 2019); generative deep neural networks (Romano et al, 2020); Metropolis-Hastings sampling (Bates et al, 2020); conditional density estimation (Tansey et al, 2021); and normalizing flows (Hansen et al, 2021). A complete review of these proposals is beyond the scope of this chapter.…”

Section: The Knockoff Frameworkmentioning

confidence: 99%

See 1 more Smart Citation

Testing conditional independence in supervised learning algorithms

Watson

Wright

2021

Mach Learn

View full text Add to dashboard Cite

We propose the conditional predictive impact (CPI), a consistent and unbiased estimator of the association between one or several features and a given outcome, conditional on a reduced feature set. Building on the knockoff framework of Candès et al. (J R Stat Soc Ser B 80:551–577, 2018), we develop a novel testing procedure that works in conjunction with any valid knockoff sampler, supervised learning algorithm, and loss function. The CPI can be efficiently computed for high-dimensional data without any sparsity constraints. We demonstrate convergence criteria for the CPI and develop statistical inference procedures for evaluating its magnitude, significance, and precision. These tests aid in feature and model selection, extending traditional frequentist and Bayesian techniques to general supervised learning tasks. The CPI may also be applied in causal discovery to identify underlying multivariate graph structures. We test our method using various algorithms, including linear regression, neural networks, random forests, and support vector machines. Empirical results show that the CPI compares favorably to alternative variable importance measures and other nonparametric tests of conditional independence on a diverse array of real and synthetic datasets. Simulations confirm that our inference procedures successfully control Type I error with competitive power in a range of settings. Our method has been implemented in an package, , which can be downloaded from https://github.com/dswatson/cpi.

show abstract

Section: The Knockoff Frameworkmentioning

confidence: 99%

“…Experiments indicate that the CRT is slightly more powerful than the ATT, but the authors caution that the former is computationally intensive and do not recommend it for large datasets. That has not stopped other groups from advancing formally similar proposals (e.g., Berrett et al, 2020;Tansey et al, 2021).…”

Section: The Knockoff Frameworkmentioning

confidence: 99%

Testing conditional independence in supervised learning algorithms

Watson

Wright

2021

Mach Learn

View full text Add to dashboard Cite

show abstract

“…However, subsequent studies showed that this procedure was less efficient than regular PC correction for dealing with fine-scale population structure 21 . Recently, a general new method, the conditional permutation test , for testing the conditional independence of variables X and Y given a potentially high dimensional random vector Z that may contain confounding factors was proposed 42 . The test permutes entries of X non-uniformly, to respect the existing dependence between X and Z and thus to account for the presence of these confounders.…”

Section: Methodsmentioning

confidence: 99%

“…The test permutes entries of X non-uniformly, to respect the existing dependence between X and Z and thus to account for the presence of these confounders. However, like the conditional randomization test of Candès et al 43 , the test relies on the availability of an approximation to the distribution of X/Z and sensitivity analysis to the misspecification of the distribution parameters showed that the method suffered from a type I error inflation increasing with the misspecification level 42 .…”

Section: Methodsmentioning

confidence: 99%

Controlling for human population stratification in rare variant association studies

Bouaziz

Mullaert

Bigio

et al. 2021

Sci Rep

View full text Add to dashboard Cite

Population stratification is a confounder of genetic association studies. In analyses of rare variants, corrections based on principal components (PCs) and linear mixed models (LMMs) yield conflicting conclusions. Studies evaluating these approaches generally focused on limited types of structure and large sample sizes. We investigated the properties of several correction methods through a large simulation study using real exome data, and several within- and between-continent stratification scenarios. We considered different sample sizes, with situations including as few as 50 cases, to account for the analysis of rare disorders. Large samples showed that accounting for stratification was more difficult with a continental than with a worldwide structure. When considering a sample of 50 cases, an inflation of type-I-errors was observed with PCs for small numbers of controls (≤ 100), and with LMMs for large numbers of controls (≥ 1000). We also tested a novel local permutation method (LocPerm), which maintained a correct type-I-error in all situations. Powers were equivalent for all approaches pointing out that the key issue is to properly control type-I-errors. Finally, we found that power of analyses including small numbers of cases can be increased, by adding a large panel of external controls, provided an appropriate stratification correction was used.

show abstract

“…Only limited extensions for non-Gaussian variables [15,31] and for nonlinear dependences [16,30] exist. Other approaches include combining a series of unconditional independence tests on the response variables (X, Y ) conditional on multiple individual values z of Z [25,17]; tests based on measures of statistical distance between estimates of the conditional densities p X|Z and p X|Y Z , which are zero if and only if X ⊥ ⊥ Y | Z [37,38]; tests based on 156 ONUR TEYMUR AND SARAH FILIPPI estimation of the conditional mutual information of X and Y given Z [19,32,33]; permutation-type tests [4,3] that require knowledge of or estimation of p X|Z ; and a large range of kernel-based methods [9,41,5,40,36] typically designed with the aim of dealing with high-dimensional or sparse problems more effectively.…”

mentioning

confidence: 99%

A Bayesian nonparametric test for conditional independence

Teymur¹,

Filippi²

2020

Foundations of Data Science

View full text Add to dashboard Cite

This article introduces a Bayesian nonparametric method for quantifying the relative evidence in a dataset in favour of the dependence or independence of two variables conditional on a third. The approach uses Pólya tree priors on spaces of conditional probability densities, accounting for uncertainty in the form of the underlying distributions in a nonparametric way. The Bayesian perspective provides an inherently symmetric probability measure of conditional dependence or independence, a feature particularly advantageous in causal discovery and not employed in existing procedures of this type.

show abstract

The Conditional Permutation Test for Independence While Controlling for Confounders

Cited by 87 publications

References 29 publications

Testing conditional independence in supervised learning algorithms

Testing conditional independence in supervised learning algorithms

Controlling for human population stratification in rare variant association studies

A Bayesian nonparametric test for conditional independence

Contact Info

Product

Resources

About