An analysis of protocols and publications suggested that most discontinuations of clinical trials were not based on preplanned interim analyses or stopping rules

Stegert, Mihaela; Kasenda, Benjamin; Elm, Erik von; You, John J.; Blümle, Anette; Tomonaga, Yuki; Saccilotto, Ramon; Amstutz, Alain; Bengough, Theresa; Briel, Matthias; Meerpohl, Joerg J; Tikkinen, Kari A.O.; Neumann, Ignacio; Carrasco‐Labra, Alonso; Faulhaber, Markus; Mulla, Sohail; Mertz, Dominik; Akl, Elie A.; Bassler, Dirk; Busse, Jason W.; Ferreira‐González, Ignacio; Lamontagne, François; Nordmann, Alain; Gloy, Viktoria; Olu, Kelechi K; Raatz, Heike; Moja, Lorenzo; Rosenthal, Rachel; Ebrahim, Shanil; Schandelmaier, Stefan; Sun, Xin; Vandvik, Per Olav; Johnston, Bradley C.; Walter, Martin A.; Burnand, Bernard; Schwenkglenks, Matthias; Hemkens, Lars G.; Bucher, Heiner C.; Guyatt, Gordon

doi:10.1016/j.jclinepi.2015.05.023

Cited by 22 publications

(17 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…While interim analyses are published quite frequently, under-reporting of this design feature exists, and therefore the actual use rate of stopping rules is greater than it appears in the published literature [2]; a particular problem is non-reporting of even the existence of a stopping rule for studies that actually complete their target total sample sizes, without the stopping rule being invoked. We acknowledge that there were only four trials with published protocols in our sample and that we did not contact trial investigators about unreported details of potentially existing stopping rules.…”

Section: Discussionmentioning

confidence: 99%

“…We included publications of randomised controlled trials (RCTs) that (1) investigated superiority of an intervention; (2) reported early stopping of the trial in the title or abstract; and (3) explicitly reported stopping for futility in the full text. Hence, we did not consider one-arm trials, non-inferiority RCTs, or the stopping of individual trial arms while the remaining trial continued, but multi-arm trials were admissible.…”

Section: Eligibility Criteria and Search Strategymentioning

confidence: 99%

“…From a survey of 894 randomised controlled trial protocols enrolling patients in the DISCOntinuation of randomised controlled trials (DISCO) study [2], it was found that 289 (32%) had pre-specified the possibility of interim analyses, and 153 (17%) had one or more stopping rules. Specifically, 38 (4.3%) protocols had specified an interim analysis for futility, and 21 out of these 38 protocols had explicitly specified a statistical stopping rule.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A systematic survey of randomised trials that stopped early for reasons of futility

Walter

Han

Guyatt

et al. 2020

BMC Med Res Methodol

View full text Add to dashboard Cite

Background: Randomised trial protocols may incorporate interim analyses, with the potential to stop the study for futility if early data show insufficient promise of a treatment benefit. Previously, we have shown that this approach will theoretically lead to mis-estimation of the treatment effect. We now wished to ascertain the importance of this phenomenon in practice. Methods: We reviewed the methods and results in a set of trials that had stopped for futility, identified through an extensive literature search. We recorded clinical areas, interventions, study design, outcomes, trial setting, sponsorship, planned and actual treatment effects, sample sizes; power; and if there was a data safety monitoring board, or a published protocol. We identified: if interim analyses were pre-specified, and how many analyses actually occurred; what pre-specified criteria might define futility; if a futility analysis formed the basis for stopping; who made the decision to stop; and the conditional power of each study, i.e. the probability of statistically significant results if the study were to continue to its complete sample size. Results: We identified 52 eligible trials, covering many clinical areas. Most trials had multiple centres, tested drugs, and 40% were industry sponsored. There were 75% where at least one interim analysis was planned a priori; a majority had only one interim analysis, typically with about half the target total sample size. A majority of trials did not pre-define a stopping rule, and a variety of reasons were given for stopping. Few studies calculated and reported low conditional power to justify the early stop. When conditional power could be calculated, it was typically low, especially under the current trend hypothesis. However, under the original design hypothesis, a few studies had relatively high conditional power. Data collection often continued after the interim analysis. Conclusions: Although other factors will typically be involved, we conclude that, from the perspective of conditional power, stopping early for futility was probably reasonable in most cases, but documentation of the basis for stopping was often missing or vague. Interpretation of truncated trials would be enhanced by improved reporting of stopping protocols, and of their actual execution.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Eligibility Criteria and Search Strategymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A systematic survey of randomised trials that stopped early for reasons of futility

Walter

Han

Guyatt

et al. 2020

BMC Med Res Methodol

View full text Add to dashboard Cite

show abstract

“…22 Given that 60% of randomised clinical trials are eventually published, 23 we estimate that 125 trials are started daily, which is 2.5 times more than the number of trial protocols registered daily (51) in 2014-2015 on clinicaltrials.gov. About 5% of registered trials are never started, 24 and many others are terminated early (12% of those posted on clinicaltrials.gov 25 ; 28% of those approved by research ethics committees). 23 Given that many trials also have secondary Might become a source of prestige to the organisations hosting these groups…”

Section: How Big Is the Task?mentioning

confidence: 99%

Outcome reporting bias in clinical trials: why monitoring matters

Ioannidis

Caplan

Dal‐Ré

2017

BMJ

View full text Add to dashboard Cite

show abstract

“…Judging this is difficult, however. Stegart et al (46) show that most trials that are stopped on the basis of either early benefit or futility have not followed stopping rules. Part of the problem here relates to the philosophy of statistics: a frequentist approach might tend to the later termination of a trial than a Bayesian one.…”

Section: Trial Conductmentioning

confidence: 99%

Ethical challenges in conducting clinical research in lung cancer

Allmark¹,

Tod²

2016

Transl. Lung Cancer Res.

View full text Add to dashboard Cite

show abstract

An analysis of protocols and publications suggested that most discontinuations of clinical trials were not based on preplanned interim analyses or stopping rules

Abstract: Two-thirds of RCT protocols did not consider interim analyses, stopping rules, or DSMBs. Most RCTs discontinued for early benefit or futility were stopped without a prespecified mechanism. When assessing trial manuscripts, journals should require access to the protocol.

Cited by 22 publications

References 19 publications

A systematic survey of randomised trials that stopped early for reasons of futility

A systematic survey of randomised trials that stopped early for reasons of futility

Outcome reporting bias in clinical trials: why monitoring matters

Ethical challenges in conducting clinical research in lung cancer

Contact Info

Product

Resources

About