A comparison of five approaches to decision‐making for a first clinical trial of efficacy

Kirby, Simon; Chuang‐Stein, Christy

doi:10.1002/pst.1775

Cited by 7 publications

(4 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The main evaluations from such a study are the drug effect estimate and whether this estimate is statistically significantly different from zero. Additional evaluations to assess commercial viability may also be included (12) but will not be the focus here.…”

Section: Introductionmentioning

confidence: 99%

Assessing Treatment Effects with Pharmacometric Models: A New Method that Addresses Problems with Standard Assessments

Chasseloup

Tessier

Karlsson

2021

AAPS J

View full text Add to dashboard Cite

Longitudinal pharmacometric models offer many advantages in the analysis of clinical trial data, but potentially inflated type I error and biased drug effect estimates, as a consequence of model misspecifications and multiple testing, are main drawbacks. In this work, we used real data to compare these aspects for a standard approach (STD) and a new one using mixture models, called individual model averaging (IMA). Placebo arm data sets were obtained from three clinical studies assessing ADAS-Cog scores, Likert pain scores, and seizure frequency. By randomly (1:1) assigning patients in the above data sets to “treatment” or “placebo,” we created data sets where any significant drug effect was known to be a false positive. Repeating the process of random assignment and analysis for significant drug effect many times (N = 1000) for each of the 40 to 66 placebo-drug model combinations, statistics of the type I error and drug effect bias were obtained. Across all models and the three data types, the type I error was (5th, 25th, 50th, 75th, 95th percentiles) 4.1, 11.4, 40.6, 100.0, 100.0 for STD, and 1.6, 3.5, 4.3, 5.0, 6.0 for IMA. IMA showed no bias in the drug effect estimates, whereas in STD bias was frequently present. In conclusion, STD is associated with inflated type I error and risk of biased drug effect estimates. IMA demonstrated controlled type I error and no bias.

show abstract

Section: Introductionmentioning

confidence: 99%

Assessing Treatment Effects with Pharmacometric Models: A New Method that Addresses Problems with Standard Assessments

Chasseloup

Tessier

Karlsson

2021

AAPS J

View full text Add to dashboard Cite

show abstract

“…Although approaches to such decision-making may vary, the confidence interval (CI) of the drug effect estimate is typically the main component, as it shows the degree of (un)certainty related to an estimate (e.g. mean difference between treatment arms) (9). Recently, different approaches under the CI framework have been compared which can be more or less conservative depending on the criteria used (9).…”

Section: Introductionmentioning

confidence: 99%

“…mean difference between treatment arms) (9). Recently, different approaches under the CI framework have been compared which can be more or less conservative depending on the criteria used (9).…”

Section: Introductionmentioning

confidence: 99%

Improved Decision-Making Confidence Using Item-Based Pharmacometric Model: Illustration with a Phase II Placebo-Controlled Trial

Llanos-Paez

Ambery

Yang

et al. 2021

AAPS J

View full text Add to dashboard Cite

This study aimed to illustrate how a new methodology to assess clinical trial outcome measures using a longitudinal item response theory–based model (IRM) could serve as an alternative to mixed model repeated measures (MMRM). Data from the EXACT (Exacerbation of chronic pulmonary disease tool) which is used to capture frequency, severity, and duration of exacerbations in COPD were analyzed using an IRM. The IRM included a graded response model characterizing item parameters and functions describing symptom-time course. Total scores were simulated (month 12) using uncertainty in parameter estimates. The 50th (2.5th, 97.5th) percentiles of the resulting simulated differences in average total score (drug minus placebo) represented the estimated drug effect (95%CI), which was compared with published MMRM results. Furthermore, differences in sample size, sensitivity, specificity, and type I and II errors between approaches were explored. Patients received either oral danirixin 75 mg twice daily (n = 45) or placebo (n = 48) on top of standard of care over 52 weeks. A step function best described the COPD symptoms-time course in both trial arms. The IRM improved precision of the estimated drug effect compared to MMRM, resulting in a sample size of 2.5 times larger for the MMRM analysis to achieve the IRM precision. The IRM showed a higher probability of a positive predictive value (34%) than MMRM (22%). An item model–based analysis data gave more precise estimates of drug effect than MMRM analysis for the same endpoint in this one case study.

show abstract

“…Frewer et al 5 proposed a frequentist decision‐making framework by comparing the boundaries of confidence interval of estimators for treatment effect with two‐level target product profiles. Kirby and Chuang‐Stein 6 compared five decision‐making approaches for clinical efficacy including hypothesis testing and single and dual‐criterion decision‐making. Roychoudhury et al 7 compared dual criterion design with hypothesis testing design and precision design for phase II trials.…”

Section: Introductionmentioning

confidence: 99%

Bayesian optimal phase II designs with dual‐criterion decision making

Zhao

Liu

et al. 2023

Pharmaceutical Statistics

View full text Add to dashboard Cite

The conventional phase II trial design paradigm is to make the go/no-go decision based on the hypothesis testing framework. Statistical significance itself alone, however, may not be sufficient to establish that the drug is clinically effective enough to warrant confirmatory phase III trials. We propose the Bayesian optimal phase II trial design with dual-criterion decision making (BOP2-DC), which incorporates both statistical significance and clinical relevance into decision making.Based on the posterior probability that the treatment effect reaches the lower reference value (statistical significance) and the clinically meaningful value (clinical significance), BOP2-DC allows for go/consider/no-go decisions, rather than a binary go/no-go decision. BOP2-DC is highly flexible and accommodates various types of endpoints, including binary, continuous, time-to-event, multiple, and coprimary endpoints, in single-arm and randomized trials. The decision rule of BOP2-DC is optimized to maximize the probability of a go decision when the treatment is effective or minimize the expected sample size when the treatment is futile. Simulation studies show that the BOP2-DC design yields desirable operating characteristics. The software to implement BOP2-DC is freely available at www.trialdesign.org.

show abstract

A comparison of five approaches to decision‐making for a first clinical trial of efficacy

Cited by 7 publications

References 11 publications

Assessing Treatment Effects with Pharmacometric Models: A New Method that Addresses Problems with Standard Assessments

Assessing Treatment Effects with Pharmacometric Models: A New Method that Addresses Problems with Standard Assessments

Improved Decision-Making Confidence Using Item-Based Pharmacometric Model: Illustration with a Phase II Placebo-Controlled Trial

Bayesian optimal phase II designs with dual‐criterion decision making

Contact Info

Product

Resources

About