Comparison of Bayesian vs Frequentist Adaptive Trial Design in the Stroke Hyperglycemia Insulin Network Effort Trial

Abstract

IMPORTANCE Bayesian adaptive trial design has the potential to create more efficient clinical trials.However, a barrier to the uptake of bayesian adaptive designs for confirmatory trials is limited experience with how they may perform compared with a frequentist design. OBJECTIVETo compare the performance of a bayesian and a frequentist adaptive clinical trial design. DESIGN, SETTING, AND PARTICIPANTSThis prospective cohort study compared 2 trial designs for a completed multicenter acute stroke trial conducted within a National Institutes of Health neurologic emergencies clinical trials network, with individual patient-level data, including the timing and order of enrollments and outcome ascertainment, from 1151 patients with acute stroke and hyperglycemia randomized to receive intensive or standard insulin therapy. The implemented frequentist design had group sequential boundaries for efficacy and futility interim analyses at 90 days after randomization for 500, 700, 900, and 1100 patients. The bayesian alternative used predictive probability of trial success to govern early termination for efficacy and futility with a first interim analysis at 500 randomized patients and subsequent interims after every 100 randomizations. MAIN OUTCOMES AND MEASURESThe main outcome was the sample size at end of study, which was defined as the sample size at which each of the studies stopped accrual of patients.RESULTS Data were collected from 1151 patients. As conducted, the frequentist design passed the futility boundary after 936 participants were randomized. Using the same sequence and timing of randomization and outcome data, the bayesian alternative crossed the futility boundary approximately 3 months earlier after 800 participants were randomized. CONCLUSIONS AND RELEVANCEBoth trial designs stopped for futility before reaching the planned maximum sample size. In both cases, the clinical community and patients would benefit from learning the answer to the trial's primary question earlier. The common feature across the 2 designs was frequent interim analyses to stop early for efficacy or for futility. Differences between how these analyses were implemented between the 2 trials resulted in the differences in early stopping.

Empirical Evaluations of Clinical Trial Designs

“…The statistical design specifies how the study adapts to the interim data, including actions such early stopping and variations of the randomization probabilities. Broglio and coauthors 1 describe what would have happened if the study team had followed the nonimplemented design. They provide the sequence of decisions that would have been made by using the data that were available for interim analyses during the SHINE clinical trial.…”

“…Broglio et al present an insightful comparison of 2 clinical trial designs: (1) the actual study design that was implemented in the Stroke Hyperglycemia Insulin Network Effort (SHINE) trial and (2) an alternative nonimplemented design. This comparison was planned before the SHINE study started enrolling patients.…”

“…Second, it illustrates that patient-level data from completed clinical trials are crucial for evaluating innovative study designs. The methodological strategy used by Broglio et al can be used in prospective or retrospective analyses of completed clinical trials to evaluate innovative aspects of other trial designs, such as the integration of pretreatment biomarkers and prognostic variables or the use of external data sets to make interim decisions during the study . These empirical evaluations are complementary to the use of simulation studies, which are frequent and useful for investigating innovative statistical designs.…”

“…Third, the article of Broglio et al suggests the exciting idea of competitions between study designs or groups of biostatisticians. Competitions in other disciplines, such as machine learning and structural biology, have provided strong incentives for developing novel methods .…”

“…The primary result of the work by Broglio et al is a point estimate of the sample size reduction of the nonimplemented bayesian design relative to the actual study design. To support the design of future clinical trials in a specific context, such as patients with hyperglycemia and acute ischemic stroke (ie, the SHINE population), both point estimates of pivotal operating characteristics (eg, sample size or the number of adverse events) and metrics of uncertainty and variability in candidate designs are necessary and equally important.…”