Multiple conditional randomization tests

Abstract

We propose a general framework for (multiple) conditional randomization tests that incorporate several important ideas in the recent literature. We establish a general sufficient condition on the construction of multiple conditional randomization tests under which their p-values are "independent", in the sense that their joint distribution stochastically dominates the product of uniform distributions under the null. Conceptually, we argue that randomization should be understood as the mode of inference precisely based on randomization. We show that under a change of perspective, many existing statistical methods, including permutation tests for (conditional) independence and conformal prediction, are special cases of the general conditional randomization test. The versatility of our framework is further illustrated with an example concerning lagged treatment effects in stepped-wedge randomized trials.

A Graph-Theoretic Approach to Randomization Tests of Causal Effects under General Interference

“…For instance, we could follow Geyer and Meeden (2005) and leverage the distribution of p-values across tests to improve power, as discussed in Basse et al (2019a). We could also adapt recent proposals on multiple randomization tests from Zhang and Zhao (2021). Second, we could investigate how much data "is thrown away" by conditioning, and suggest biclique decompositions of the null exposure graph that minimize data loss.…”

What is a randomization test?

“…In Section 3 we have described three perspectives on conditioning in randomization tests: conditioning on a set of treatment assignments, conditioning on a σ-algebra, and conditioning on a random variable. They are useful for different purposes: the first perspective is useful when the null hypothesis is partially sharp and one needs to construct a computable p-value; the second perspective is useful to describe the conditioning structure of multiple CRTs [67]; and the third perspective is the most interpretable and allows us to consider post-experimental randomization. We have collected some useful methods to construct CRTs in Section 4.…”

“…This measure-theoretic formulation is useful for extending the theory above to continuous treatments and consider the structure of conditioning events in multiple CRTs, which is considered in an accompanying methodological article [67].…”