Frequentist operating characteristics of Bayesian optimal designs via simulation

Abstract

Bayesian adaptive designs have become popular because of the possibility of increasing the number of patients treated with more beneficial treatments, while still providing sufficient evidence for treatment efficacy comparisons. It can be essential, for regulatory and other purposes, to conduct frequentist analyses both before and after a Bayesian adaptive trial, and these remain challenging. In this paper, we propose a general simulation-based approach to compare frequentist designs with Bayesian adaptive designs based on frequentist criteria such as power and to compute valid frequentist p-values. We illustrate our approach by comparing the power of an equal randomization (ER) design with that of an optimal Bayesian adaptive (OBA) design. The Bayesian design considered here is the dynamic programming solution of the optimization of a specific utility function defined by the number of successes in a patient horizon, including patients whose treatment will be affected by the trial's results after the end of the trial.While the power of an ER design depends on treatment efficacy and the sample size, the power of the OBA design also depends on the patient horizon size. Our results quantify the trade-off between power and the optimal assignment of patients to treatments within the trial. We show that, for large patient horizons, the two criteria are in agreement, while for small horizons, differences can be substantial. This has implications for precision medicine, where patient horizons are decreasing as a result of increasing stratification of patients into subpopulations defined by molecular markers. KEYWORDSBayesian adaptive designs, dynamic programming, frequentist analyses, optimal strategy Spiegelhalter et al 5 and Cellamare et al 6 have illustrated the use of Bayesian approaches in clinical trials and discussed their advantages. Ethical concerns 7 associated with adaptive designs are also relevant in Bayesian adaptive settings. On the other hand, frequentist criteria remain the mainstay in regulatory decision-making and in the medical literature. This applies to both trial design requirements, such as the power at a given type I error level, and reported 4026

Bayesian Network Analysis for Prediction of Unplanned Hospital Readmissions of Cancer Patients with Breakthrough Cancer Pain and Complex Care Needs

“…For this aim, a Bayesian network approach can be useful as it agrees with a logic structure from the data and allows optimal prediction combined with a useful causal inference process. As we showed, this approach can enable the learning of reliable structures in the context of causal relations [ 27 ]. Among the Bayesian networks, the BIC-based estimated model can develop certain relationships that are a priori established by the clinician.…”

“…It is linked to the likelihood of the model regarding the estimated parameters and contains associations between variables. Theoretical models are validated according to the BIC minimization or other indicators (e.g., Akaike Information Criteria, AIC, Bayesian Dirichlet Equivalent) [ 27 ]. Choice of the BIC model due to observation penalization balance (its reliability decreases as the number of observations increases) and implementation of the Bayesian network for exact inference [ 28 ].…”

“…It is linked to the likelihood of the model regarding the estimated parameters and contains associations between variables. Theoretical models are validated according to the BIC minimization or other indicators (e.g., Akaike Information Criteria, AIC, Bayesian Dirichlet Equivalent) [ 27 ].…”

Generalisations of a Bayesian decision-theoretic randomisation procedure and the impact of delayed responses

“…Since the size of T influences how the procedure attains a compromise between the power and patient benefit trade-off, and contributes to the optimality criterion, it is therefore an important element in the specification of such procedures (see e.g. Upton and Lee, 1976 ; Zhang et al, 2019 ).…”

Response-adaptive randomization in clinical trials: from myths to practical considerations

“…the stopping boundaries) can then be adjusted to satisfy the type I error constraints. Variations of this approach have been used in Wason and Trippa (2014); Wathen and Thall (2017); Zhang et al (2019) for example, all in the context of calibrating multi-arm Bayesian RAR procedures to have correct type I error control. Applying this approach can be computationally intensive however.…”