Addressing potential prior‐data conflict when using informative priors in proof‐of‐concept studies

Abstract

Bayesian methods are increasingly used in proof-of-concept studies. An important benefit of these methods is the potential to use informative priors, thereby reducing sample size. This is particularly relevant for treatment arms where there is a substantial amount of historical information such as placebo and active comparators. One issue with using an informative prior is the possibility of a mismatch between the informative prior and the observed data, referred to as prior-data conflict. We focus on two methods for dealing with this: a testing approach and a mixture prior approach. The testing approach assesses prior-data conflict by comparing the observed data to the prior predictive distribution and resorting to a non-informative prior if prior-data conflict is declared. The mixture prior approach uses a prior with a precise and diffuse component. We assess these approaches for the normal case via simulation and show they have some attractive features as compared with the standard one-component informative prior. For example, when the discrepancy between the prior and the data is sufficiently marked, and intuitively, one feels less certain about the results, both the testing and mixture approaches typically yield wider posterior-credible intervals than when there is no discrepancy. In contrast, when there is no discrepancy, the results of these approaches are typically similar to the standard approach. Whilst for any specific study, the operating characteristics of any selected approach should be assessed and agreed at the design stage; we believe these two approaches are each worthy of consideration.

Predictive probability of success using surrogate endpoints

“…A prior‐data conflict is declared if lies outside the ξ 100th and (1 − ξ )100th percentiles of this prior predictive distribution, ie, if where ξ is a predefined testing level, usually 5% or 10% . It could be chosen by computing the operational characteristics of simulated data scenarios, to ensure that clinically pertinent prior‐data conflicts are detected with this approach.…”

“…The choice of is therefore critical in the analysis and could be based on operational characteristics of different simulated scenarios. Nevertheless, Mutsvari et al point out that “the rationale for using a mixture prior will be strongest when there is some justification for each of the two components” and w .…”

“…The sensitivity of the prior‐data conflict approaches to the testing level ξ , the prior probability w , and the vague prior component was assessed in the work of Mutsvari et al We added simulation scenarios in our context in the Supple mental Material, Section 1, in order to assess the impact of the choice of ξ for the testing approach and of w for the mixture approach on the PPoS calculation. First, for both approaches, handling prior‐data conflict does not strongly affect the results and does not change the conclusions when there is no conflict in reality, whatever the choice of the parameters.…”

“…For example, we could be interested to continue the development if an overwhelming efficacy is observed on the final endpoint, even if no effect is observed on the surrogate. We propose two approaches to handle this issue in Sections 2.3.2 and 2.3.3, using the methods for addressing potential prior-data conflict presented in the work of Mutsvari et al 35 The principle of the first approach, referred to as the "testing approach," is to assess the level of conflict between the two sources of evidence and to discard the information coming from the surrogate endpoint if this level of conflict exceeds a certain predefined threshold. On the other hand, the second approach, referred to as the "mixture prior approach," always consider both sources of evidence in the predictions but permits to down weight the information coming from the surrogate endpoint in case of conflict.…”

Power priors based on multiple historical studies for binary outcomes

“…If more than one relevant source of historical information is available, then all of these should be incorporated into the calculations, however not all of the methods can easily be extended to handle many sources. Mutsvari, Tytgat, and Walley (2015) describe the problem of prior-data conflict that can occur when including historical data and describe approaches to prevent them. van Rosmalen, Dejardin, Yvette van Norden, Löwenberg, and Lesaffre (2017) do a broad study of methods for incorporating multiple sources of historical data into a clinical trial analysis, looking at a time-to-event endpoint.…”