Impact of baseline covariate imbalance on bias in treatment effect estimation in cluster randomized trials: Race as an example

Abstract

A R T I C L E I N F O Keywords: (MeSH terms): Bias Cluster analysis Multivariable analysis Randomized controlled trials as topic Research design A B S T R A C TIndividual-level baseline covariate imbalance could happen more frequently in cluster randomized trials, and may influence the observed treatment effect. Using computer and real-data simulations, this paper quantifies the extent and impact of covariate imbalance on the estimated treatment effect for both continuous and binary outcomes, and relates it to the degree of imbalance for different numbers of clusters, cluster sizes, and covariate intraclass correlation coefficients. We focused on the impact of race as a covariate, given the emphasis of regulatory and funding bodies on understanding the influence of demographic characteristics on treatment effectiveness. We found that bias in the treatment effect is proportional to both the degree of baseline covariate imbalance and the covariate effect size. Larger numbers of clusters result in lower covariate imbalance, and increasing cluster size is less effective in reducing imbalance compared to increasing the number of clusters. Models adjusted for important baseline confounders are superior to unadjusted models for minimizing bias in both model-based simulations and an innovative simulation based on real clinical trial data. Higher outcome intraclass correlation coefficients did not affect bias but resulted in greater variance in treatment estimates. outcomes) or odds ratio (for binary outcomes) suffices as a measure of treatment effect. In practice, results for many CRTs have also been reported using similar (unadjusted) estimators [4,5]. However, the internal validity of CRTs is challenged when treatment arms are not comparable. Brierley et al. [6] found that about 30% of CRTs may suffer from selection bias, resulting in substantial covariate imbalance. Previous literature recommended that researchers could control selection bias in CRTs via cautious design, such as identifying patients prior to randomization and blinding [7][8][9]. Aside from selection bias, another important source of covariate imbalance is random chance, and that is the focus of the current paper. We show that baseline covariate imbalance may happen more frequently in CRTs compared to RCTs. If the covariates are predictive of the clinical outcomes, they act as confounders and may cause a spurious association between intervention and clinical outcome when not accounted for properly. As a result, it is important to understand how to accurately account for important individual-level covariates in the design and analysis of CRTs where confounders typically can impact the outcome.

Optimizing Therapy to Prevent Avoidable Hospital Admissions in Multimorbid Older Adults (OPERAM): cluster randomised controlled trial

“…In prespecified sensitivity analyses we adjusted for baseline characteristics and investigated time variation of the intervention effect 51. A post hoc added sensitivity analysis only considered data collected in interviews conducted within protocol specified time windows.…”

Sample size requirements for detecting treatment effect heterogeneity in cluster randomized trials

“…This same model has also been discussed in Spybrook et al, 23 where X ij is binary. Of note, model () is a direct extension of those studied in Raudenbush, 29 Li et al, 30 and Yang et al, 31 where only main effects of W i and X ij are considered. Further extensions of model () to allow for random coefficients for X ij can be found in Jaciw et al 32 and Dong et al 24 …”

“…In the study, patients recruited from 82 sites (heart clinics or heart and vascular centers) were randomized to receive either usual care plus aerobic exercise training, or usual care alone. In Yang et al, 31 we have previously used the outcome and covariate data from the HF‐ACTION study to recreate a CRT to assess the bias in estimating the OTE due to baseline imbalance. In this section, we use the same context and baseline covariate data from HF‐ACTION to inform the design parameters and exemplify how to estimate the required sample size and power for testing HTE, were the investigators to conduct a CRT using the HF‐ACTION population.…”

Comparison of two propensity score-based methods for balancing covariates: the overlap weighting and fine stratification methods in real-world claims data

“…After balancing covariates, we determined relative performance of OW and FS for model bias and precision. The degree of covariate imbalance is proportional to bias in the treatment effect [ 24 ], and final sample size is associated with precision. However, due to lack of knowledge of the true FQHC effect in the empirical example, we do not know the real size of model bias and precision, especially, in a setting of infrequent exposure/outcome.…”