Stratification Trees for Adaptive Randomization in Randomized Controlled Trials

Abstract

This paper proposes an adaptive randomization procedure for two-stage randomized controlled trials. The method uses data from a first-wave experiment in order to determine how to stratify in a second wave of the experiment, where the objective is to minimize the variance of an estimator for the average treatment effect (ATE). We consider selection from a class of stratified randomization procedures which we call stratification trees: these are procedures whose strata can be represented as decision trees, with differing treatment assignment probabilities across strata. By using the first wave to estimate a stratification tree, we simultaneously select which covariates to use for stratification, how to stratify over these covariates, as well as the assignment probabilities within these strata. Our main result shows that using this randomization procedure with an appropriate estimator results in an asymptotic variance which minimizes the variance bound for estimating the ATE, over an optimal stratification of the covariate space. Moreover, by extending techniques developed in Bugni et al. (2018), the results we present are able to accommodate a large class of assignment mechanisms within strata, including stratified block randomization. We also present extensions of the procedure to the setting of multiple treatments, and to the targeting of subgroup-specific effects. In a simulation study, we find that our method is most effective when the response model exhibits some amount of "sparsity" with respect to the covariates, but can be effective in other contexts as well, as long as the first-wave sample size used to estimate the stratification tree is not prohibitively small. We conclude by applying our method to the study in Karlan and Wood (2017), where we estimate stratification trees using the first wave of their experiment.

Asymptotic Efficiency Bounds for a Class of Experimental Designs

“…Using data from this experimental design, one can achieve the semiparametric efficiency bound v e * () using an estimator that adjusts flexibly for covariates or uses a flexible estimate of the propensity score (Hahn et al, 2011). To avoid the additional complexity of such estimators, one can alternatively design the experiment using stratification on covariates, so that a simple estimator that weights on the (true) propensity score achieves the bound (Tabord-Meehan, 2018;Cytrynbaum, 2021).…”

“…In the case of a binary treatment, the efficiency bound of Hahn (1998) gives a lower bound on the asymptotic performance of estimators and tests for the average treatment effect (ATE) under experimental designs that lead to independent and identically distributed (iid) data. A key finding is that one can use data from past waves to design an experiment that optimizes this bound, along with a subsequent estimator that achieves the optimized bound (Hahn et al, 2011;Tabord-Meehan, 2018;Cytrynbaum, 2021).…”

Bootstrap Inference for Quantile Treatment Effects in Randomized Experiments with Matched Pairs

“…Therefore, we use fundamentally different asymptotic arguments and bootstrap methods from those employed by Zhang and Zheng (2020). Our paper also fits in the growing literature of studying the inference in randomized experiments, e.g., Hahn, Hirano, and Karlan (2011), Athey and Imbens (2017), Abadie, Chingos, and West (2018), Bugni, Canay, and Shaikh (2018), Tabord-Meehan (2018), and Bugni, Canay, and Shaikh (2019), among others.…”

Designing Experiments Toward Shrinkage Estimation

“…While Thompson sampling was originally designed for a more generic problem involving maximizing the expected reward, it can be used to estimate average stratum treatment effects, as discussed in Offer-Westort et al (2021). Adaptive experimental design is an area of active research, though much recent work has focused on methods that define strata in the second phase, rather than taking strata as fixed (Tabord-Meehan, 2018;Bai, 2019). For modern methods that incorporate a fixed stratification scheme, see Hahn et al (2011) and Chambaz et al (2014).…”