Alternative Views On Setting Clinical Trial Futility Criteria

Abstract

A feature increasingly utilized in clinical trial practice is to allow a study to stop early when it seems unlikely to achieve its primary efficacy objectives. This is commonly referred to as stopping for futility, and can be motivated by ethical and financial considerations. A number of methods for addressing futility have been described in the literature, including rules based upon conditional power, predictive probability, beta spending functions, and others. We consider futility stopping from the point of view of quantifying and providing an objective sensible balance between risks of incorrect decisions (e.g., stopping trials which should continue, and continuing trials which should stop), and discuss how specific considerations within a trial can lead to choice of a sensible scheme. This approach is not specific to any particular scales in the literature such as those just mentioned, and we describe interrelationships among criteria expressed on different scales. As futility may be evaluated multiple times in a long-term trial and the amount of information available at scheduled interim analyses may be difficult to predict in advance, we present a specific optimality criterion and discuss which of the familiar scales tend to produce schemes simple to describe and implement, and with better behavior across different timepoints at which futility might be evaluated.

The Adaptive designs CONSORT Extension (ACE) statement: a checklist with explanation and elaboration guideline for reporting randomised trials that use an adaptive design

“…Early stopping of a trial or treatment group for futility can be evaluated based on statistical information to support lack of evidence of benefit that is derived and expressed in several ways. For example, conditional power,56147148149150 predictive power,55148151152153 the threshold of the treatment effect, posterior probability of the treatment effect,100 or some form of clinical utility that quantifies the balance between benefits against harms154155 or between patient and society perspectives on health outcomes 100. See box 8 for exemplars.…”

Decision‐making in drug development using a composite definition of success

“…One may prefer to use a frequentist framework instead of a Bayesian one where only vague priors are used and to present the same success component criteria on different scales such as standardized differences or conditional powers. 49 These are common approaches when the success definition is based solely on the primary efficacy endpoint, but some difficulty arises when trying to derive a single frequentist test statistic on multiple outcomes of benefit and risk, which often have different distributions.…”

“…Depending on the therapeutic area and the phase of development, some thresholds could be defined using prespecified targeted levels of evidence following, for example, the concepts developed by Neuenschwander et al 53 or Frewer et al 8 In any case, one should be careful in making decisions based on a direct, intuitive, interpretation of the PPoS as a chance of success for the development. 49 The probability of composite success presented here rather corresponds to a "probability of technical success [ … ] defined as the probability of a compound generating favorable data to support a filing to regulators" 54 and supports decision in favor of one development strategy when the whole set of results (on the three components and the composite, for the main analysis and the sensitivity analyses) supports the belief of a positive outcome.…”

The Adaptive designs CONSORT Extension (ACE) Statement: A checklist with explanation and elaboration guideline for reporting randomised trials that use an adaptive design

“…Early stopping of a trial or treatment group for futility can be evaluated based on statistical information to support lack of evidence of benefit that is derived and expressed in several ways. For example, conditional power [57,[153][154][155][156], predictive power [56,154,[157][158][159], the threshold of the treatment effect, posterior probability of the treatment effect [105], or some form of clinical utility that quantifies the balance between benefits against harms [160,161] or between patient and society perspectives on health outcomes [105].…”

“…Thus, nonbinding decision rules are those that can be overruled without having a negative effect on the control of the type I error rate. Use of non-binding futility boundaries is often advised [56].…”