Cohort Multiple Randomised Controlled Trials (cmRCT) design: efficient but biased? A simulation study to evaluate the feasibility of the Cluster cmRCT design

Abstract

BackgroundThe Cohort Multiple Randomised Controlled Trial (cmRCT) is a newly proposed pragmatic trial design; recently several cmRCT have been initiated. This study tests the unresolved question of whether differential refusal in the intervention arm leads to bias or loss of statistical power and how to deal with this.MethodsWe conduct simulations evaluating a hypothetical cluster cmRCT in patients at risk of cardiovascular disease (CVD). To deal with refusal, we compare the analysis methods intention to treat (ITT), per protocol (PP) and two instrumental variable (IV) methods: two stage predictor substitution (2SPS) and two stage residual inclusion (2SRI) with respect to their bias and power. We vary the correlation between treatment refusal probability and the probability of experiencing the outcome to create different scenarios.ResultsWe found ITT to be biased in all scenarios, PP the most biased when correlation is strong and 2SRI the least biased on average. Trials suffer a drop in power unless the refusal rate is factored into the power calculation.ConclusionsThe ITT effect in routine practice is likely to lie somewhere between the ITT and IV estimates from the trial which differ significantly depending on refusal rates. More research is needed on how refusal rates of experimental interventions correlate with refusal rates in routine practice to help answer the question of which analysis more relevant. We also recommend updating the required sample size during the trial as more information about the refusal rate is gained.

The Trials within Cohorts design faced methodological advantages and disadvantages in the exercise oncology setting

“…During the recruitment phase, the actual acceptance rate turned out to be lower than expected, and we had to increase our sample size and extend our recruitment period [11,32]. Fortunately, UMBRELLA Fit was performed within a dynamic cohort with continuous recruitment of eligible patients.…”

Erratum to: Evaluation of biases present in the cohort multiple randomised controlled trial design: a simulation study

“…The published Reference 14 [2] is incorrect, and should have cited a different article by Pate et al [3]. …”

“…Pate et al [3] should have been included in the Reference list as 14, and the subsequent references (and citations within the article text) should have been renumbered.…”

The cohort multiple randomized controlled trial design was found to be highly susceptible to low statistical power and internal validity biases

“…Some authors have described cmRCTs as producing an estimate of the intention-to-treat (ITT) effect (effect of accepting treatment [25] or effect of accepting and fully complying with treatment [26] ), albeit an inherently biased estimate in the presence of nonconsent. However, in point of fact, the design compares all patients randomly selected to be offered treatment with those not selected, hence—ignoring for now issues of preselection into the cohort itself—the effect being estimated is best characterized as the mean effect of being selected for treatment, or assuming the percentage of noncontactable patients to be suitably low, the mean effect of the offer of treatment regardless of whether that offer is taken up or not.…”

“…In view of this impact of nonconsent on the estimate of effect, instrumental variable (IV) and complier average causal effect (CACE) methods have been proposed as means for estimating the mean effect for patients who consent to treatment only [25] , [26] , [27] . These methods generally produce larger effect estimates: CACE, for example, is the offer-of-treatment effect divided by the rate of consent [28] and will, for instance, double the estimate of effect when consent is 50%.…”