Analysis of Covariance in Randomized Trials: More Precision and Valid Confidence Intervals, Without Model Assumptions

Abstract

Covariate adjustment" in the randomized trial context refers to an estimator of the average treatment effect that adjusts for chance imbalances between study arms in baseline variables (called "covariates"). The baseline variables could include, for example, age, sex, disease severity, and biomarkers. According to two surveys of clinical trial reports, there is confusion about the statistical properties of covariate adjustment. We focus on the analysis of covariance (ANCOVA) estimator, which involves fitting a linear model for the outcome given the treatment arm and baseline variables, and trials that use simple randomization with equal probability of assignment to treatment and control.We prove the following new (to the best of our knowledge) robustness property of ANCOVA to arbitrary model misspecification: Not only is the ANCOVA point estimate consistent (as proved by Yang and Tsiatis, 2001) but so is its standard error. This implies that confidence intervals and hypothesis tests conducted as if the linear model were correct are still asymptotically valid even when the linear model is arbitrarily misspecified, for example, when the baseline variables are nonlinearly related to the outcome or there is treatment effect heterogeneity. We also give a simple, robust formula for the variance reduction (equivalently, sample size reduction) from using ANCOVA. By reanalyzing completed randomized trials for mild cognitive impairment, schizophrenia, and depression, we demonstrate how ANCOVA can achieve variance reductions of 4 to 32%. K E Y W O R D Simbalance, relative efficiency, robustness

Rejoinder to “Robustness of ANCOVA in randomized trials with unequal randomization” by Jonathan W. Bartlett

“…Though the results of Wang et al . () and Bartlett () only apply under simple randomization, we conjecture that analogous results hold for some response‐adaptive randomization schemes that target an optimal allocation. Robust variance estimators will typically be needed.…”

“…The results of Wang et al . () and Bartlett () can be extended to incorporate adjustment for short‐term outcomes as well as baseline variables, using estimators (different from ANCOVA) that are robust to model misspecification (see, eg, Qian et al ., ). Robust variance estimators are required.…”

Robustness of ANCOVA in randomized trials with unequal randomization

“…Following the notation of Wang et al . (), we assume we observe n independent and identically distributed copies of , where is a column vector of bounded baseline covariates, A is the binary treatment group indicator ( for experimental treatment, for control) and Y is the continuous outcome. Like Wang et al .…”

“…Recently in the journal Wang et al . () proved that the model‐based variance estimator from an ANCOVA analysis of a randomized trial is valid under arbitrary misspecification, and therefore advocated its use for analysis of trials with continuous outcomes. Concurrently, the US Food and Drug Administration () has recently issued draft guidance on the topic of baseline covariate adjustment in randomized trials with continuous outcomes.…”

“…Wang et al . () assumed that the probability of randomization to each arm is 1/2 and stated that if this is not the case, a robust variance estimator should be used. While a randomization probability of 1/2 is most common, many trials are conducted with unequal randomization probabilities.…”