Inference Under Covariate-Adaptive Randomization

Abstract

Standard-Nutzungsbedingungen:Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden.Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen.Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Abstract This paper studies inference for the average treatment effect in randomized controlled trials with covariate-adaptive randomization. Here, by covariate-adaptive randomization, we mean randomization schemes that first stratify according to baseline covariates and then assign treatment status so as to achieve "balance" within each stratum. Such schemes include, for example, Efron's biased-coin design and stratified block randomization. When testing the null hypothesis that the average treatment effect equals a pre-specified value in such settings, we first show that the usual two-sample t-test is conservative in the sense that it has limiting rejection probability under the null hypothesis no greater than and typically strictly less than the nominal level. In a simulation study, we find that the rejection probability may in fact be dramatically less than the nominal level. We show further that these same conclusions remain true for a naïve permutation test, but that a modified version of the permutation test yields a test that is non-conservative in the sense that its limiting rejection probability under the null hypothesis equals the nominal level. The modified version of the permutation test has the additional advantage that it has rejection probability exactly equal to the nominal level for some distributions satisfying the null hypothesis. Finally, we show that the usual t-test (on the coefficient on treatment assignment) in a linear regression of outcomes on treatment assignment and indicators for each of the strata yields a nonconservative test as well. In a simulation study, we find that the non-conservative tests have substantially greater power than the usual two-sample t-test. Terms of use: Documents in

Analysis of Covariance in Randomized Trials: More Precision and Valid Confidence Intervals, Without Model Assumptions

“…As in their work, our results are asymptotic, that is, they hold in the limit as sample size grows to infinity while the set of covariates is fixed. A special case of the results of Bugni et al () coincides with a special case of our result in Section (explained in that section).…”

“…Bugni et al () focused on trials using covariate‐adaptive randomization, but their results also have implications for simple randomization as considered here. In particular, the special case of the above theorem where is a single, categorical variable follows from Theorem 4.3 and Remark 4.6 of Bugni et al ().…”

“…This assumption does not hold if stratified randomization or covariate‐adaptive randomization is used. For stratified randomization and some types of covariate‐adaptive randomization, Bugni et al () showed that if the covariates in the ANCOVA model are the indicators of the strata used in the randomization procedure, then the ANCOVA estimator is consistent; furthermore, its model‐based variance estimator is consistent if the limiting probability of assignment to each arm is within each stratum. The previous sentence holds regardless of whether the true data generating distribution satisfies any of the ANCOVA model assumptions.…”

Inference in Experiments with Matched Pairs

“…For instance, within each stratum, a researcher may assign (uniformly) at random half of the units to treatment and the remainder to control. Bugni et al (2018Bugni et al ( , 2019 develop a variety of results pertaining to these ways of assigning treatment status, but their analysis relies heavily upon the requirement that the units are stratified using the baseline covariates into only a finite number of strata. As a result, their framework cannot accomodate "matched pairs" designs, where the number of strata is equal to the number of pairs and therefore proportional to the sample size.…”

“…A growing literature suggests that it should be possible to achieve limiting rejection probability under the null hypothesis equal to α by studentizing the test statistic using a consistent estimator of (19). See, for example, Janssen (1997), Chung and Romano (2013), DiCiccio and Romano (2017) and Bugni et al (2018).…”