Idle thoughts of a ‘well‐calibrated’ Bayesian in clinical drug development

Grieve, Andy

doi:10.1002/pst.1736

Cited by 34 publications

(49 citation statements)

References 113 publications

(125 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, it seems to be hidden knowledge in the Bayesian community that no power gain is possible when type I error needs to be controlled, which has been stated before by, for example, Psioda and Ibrahim (): “If one wishes to control the type I error rate in the traditional frequentist sense, all prior information must be disregarded in the analysis.” These authors also give a formal proof in case of the one‐sample one‐sided test of a normal endpoint in the context of power priors with fixed power parameter, that is, a situation where the same amount of prior information is incorporated independent of the data. Similarly, Grieve (), again in the context of constant borrowing of information, acknowledges, referring to FDA and CDRH (): “[...] requiring strict control of the type I error results in 100% discounting of the prior information. [...] This [...] is important in the context of the remark in the FDA's Bayesian guidance that ‘it may be appropriate to control the type I error at a less stringent level than when no prior information is used’.…”

Section: Introductionmentioning

confidence: 99%

Power gains by using external information in clinical trials are typically not possible when requiring strict type I error control

2019

View full text Add to dashboard Cite

In the era of precision medicine, novel designs are developed to deal with flexible clinical trials that incorporate many treatment strategies for multiple diseases in one trial setting. This situation often leads to small sample sizes in disease‐treatment combinations and has fostered the discussion about the benefits of borrowing of external or historical information for decision‐making in these trials. Several methods have been proposed that dynamically discount the amount of information borrowed from historical data based on the conformity between historical and current data. Specifically, Bayesian methods have been recommended and numerous investigations have been performed to characterize the properties of the various borrowing mechanisms with respect to the gain to be expected in the trials. However, there is common understanding that the risk of type I error inflation exists when information is borrowed and many simulation studies are carried out to quantify this effect. To add transparency to the debate, we show that if prior information is conditioned upon and a uniformly most powerful test exists, strict control of type I error implies that no power gain is possible under any mechanism of incorporation of prior information, including dynamic borrowing. The basis of the argument is to consider the test decision function as a function of the current data even when external information is included. We exemplify this finding in the case of a pediatric arm appended to an adult trial and dichotomous outcome for various methods of dynamic borrowing from adult information to the pediatric arm. In conclusion, if use of relevant external data is desired, the requirement of strict type I error control has to be replaced by more appropriate metrics.

show abstract

Section: Introductionmentioning

confidence: 99%

Power gains by using external information in clinical trials are typically not possible when requiring strict type I error control

2019

View full text Add to dashboard Cite

show abstract

“…The cut-points would have to change if we used vague priors. If the subgroup data are not consistent with the informative priors, the result could result in decreased power, particularly if δ values are not adjusted [6]. …”

Section: Discussionmentioning

confidence: 99%

“…Multiplicity adjustment is required to preserve the overall Type I error rate [5]. We calibrated [6] the operating characteristics in simulations to ensure the overall Type I error rate was close to 0.05 (one-sided) in all designs that used different statistical models.…”

Section: Introductionmentioning

confidence: 99%

Personalized medicine enrichment design for DHA supplementation clinical trial

Lei

Mayo

Carlson

et al. 2017

Contemporary Clinical Trials Communications

View full text Add to dashboard Cite

Personalized medicine aims to match patient subpopulation to the most beneficial treatment. The purpose of this study is to design a prospective clinical trial in which we hope to achieve the highest level of confirmation in identifying and making treatment recommendations for subgroups, when the risk levels in the control arm can be ordered. This study was motivated by our goal to identify subgroups in a DHA (docosahexaenoic acid) supplementation trial to reduce preterm birth (gestational age<37 weeks) rate. We performed a meta-analysis to obtain informative prior distributions and simulated operating characteristics to ensure that overall Type I error rate was close to 0.05 in designs with three different models: independent, hierarchical, and dynamic linear models. We performed simulations and sensitivity analysis to examine the subgroup power of models and compared results to a chi-square test. We performed simulations under two hypotheses: a large overall treatment effect and a small overall treatment effect. Within each hypothesis, we designed three different subgroup effects scenarios where resulting subgroup rates are linear, flat, or nonlinear. When the resulting subgroup rates are linear or flat, dynamic linear model appeared to be the most powerful method to identify the subgroups with a treatment effect. It also outperformed other methods when resulting subgroup rates are nonlinear and the overall treatment effect is big. When the resulting subgroup rates are nonlinear and the overall treatment effect is small, hierarchical model and chi-square test did better. Compared to independent and hierarchical models, dynamic linear model tends to be relatively robust and powerful when the control arm has ordinal risk subgroups.

show abstract

“…Although Bayesian methods do not rely on the frequentist paradigm of repeated testing, it is still useful to test the frequentist operating characteristics of Bayesian methods (Rubin, 1984;Grieve, 2016). Unlike frequentist methods, whose operating characteristics are often well defined by construction, Bayesian methods are more flexible and so simulation studies must be used to determine how they perform.…”

Section: Operating Characteristics Definitionsmentioning

confidence: 99%

“…While a large increase is not acceptable, Grieve (2016) argues that a small increase should not prevent the use of Bayesian methods. Bayesian methods with informative priors necessarily have some increase in type I error probability compared to standard frequentist methods.…”

Section: Type I Errormentioning

confidence: 99%

Power priors based on multiple historical studies for binary outcomes

Gravestock

Held

2018

Biometrical J

View full text Add to dashboard Cite

Incorporating historical information into the design and analysis of a new clinical trial has been the subject of much recent discussion. For example, in the context of clinical trials of antibiotics for drug resistant infections, where patients with specific infections can be difficult to recruit, there is often only limited and heterogeneous information available from the historical trials. To make the best use of the combined information at hand, we consider an approach based on the multiple power prior that allows the prior weight of each historical study to be chosen adaptively by empirical Bayes. This choice of weight has advantages in that it varies commensurably with differences in the historical and current data and can choose weights near 1 if the data from the corresponding historical study are similar enough to the data from the current study. Fully Bayesian approaches are also considered. The methods are applied to data from antibiotics trials. An analysis of the operating characteristics in a binomial setting shows that the proposed empirical Bayes adaptive method works well, compared to several alternative approaches, including the meta-analytic prior.

show abstract

Idle thoughts of a ‘well‐calibrated’ Bayesian in clinical drug development

Cited by 34 publications

References 113 publications

Power gains by using external information in clinical trials are typically not possible when requiring strict type I error control

Power gains by using external information in clinical trials are typically not possible when requiring strict type I error control

Personalized medicine enrichment design for DHA supplementation clinical trial

Power priors based on multiple historical studies for binary outcomes

Contact Info

Product

Resources

About