The Sceptical Bayes Factor for the Assessment of Replication Success

Pawel, Samuel; Held, Leonhard

doi:10.1111/rssb.12491

Cited by 15 publications

(27 citation statements)

References 75 publications

(163 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A similar procedure but using AnCred based on Bayes factors as in Section 3 was proposed in Pawel and Held. 29 Reverse‐Bayes inference seems to fit naturally into this setting as it provides a formal framework to challenge and substantiate scientific findings.…”

Section: Discussionmentioning

confidence: 99%

“… 24 , 25 , 26 The appeal of Reverse‐Bayes techniques has recently been widened by the development of inferential methods using both posterior probabilities and Bayes factors. 18 , 29 …”

Section: Discussionmentioning

confidence: 99%

“…We could again use the Box 11 prior‐predictive check as in Section 2.2, however, the resulting tail probability is difficult to compare to the Bayes‐factor cut‐off

γ .

When a specific alternative model to the null is in mind, Box 11 (p391) also suggested to use a Bayes factor for model criticism of the null model. Following this approach, Pawel and Held 29 proposed to define a second Bayes factor contrasting the sufficiently sceptical prior to an optimistic prior, which they defined as

θ ∣ H_{2} \sim normalN ((),, \overset{truê}{θ}, σ^{2})

the posterior of

θ

based on the data and the reference prior

f ((), θ) \propto 1

. The optimistic prior therefore represents the position of a proponent who takes the original claim at face value.…”

Section: Reverse‐bayes Methods With Bayes Factorsmentioning

confidence: 99%

“…Most published work has focused on their use in putting new research claims in context, with Reverse‐Bayes methods being used to assess whether the prior evidence needed to make a claim credible is consistent with existing insight. 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 …”

Section: Introduction: the Origin Of Reverse‐bayes Methodsmentioning

confidence: 99%

“…Intrinsic credibility at level

is established when,

and we are interested in the Bayes factor for intrinsic credibility

which is the smallest level

where ( A1 ) holds. The

is therefore a special case of the sceptical Bayes factor from Pawel and Held 29 where the same data is used in both Bayes factors (instead of the data from a replication study for

It is hence given by,

with

, and

the branch of the Lambert

function that satisfies

1

for

2.04

is located at the intersection between

and

in the relative prior variance g , so Equation ( 7 ) from Pawel and Held 29 can be used. For

0.83

remains below

for all

and hence

is given by

the minimum of

from ( 16 ).…”

Section: Mean Of the Advocacy Priormentioning

confidence: 99%

See 4 more Smart Citations

Reverse‐Bayes methods for evidence assessment and research synthesis

Held

Matthews

Ott

et al. 2021

Research Synthesis Methods

Self Cite

View full text Add to dashboard Cite

It is now widely accepted that the standard inferential toolkit used by the scientific research community-null-hypothesis significance testing (NHST)-is not fit for purpose. Yet despite the threat posed to the scientific enterprise, there is no agreement concerning alternative approaches for evidence assessment. This lack of consensus reflects long-standing issues concerning Bayesian methods, the principal alternative to NHST. We report on recent work that builds on an approach to inference put forward over 70 years ago to address the well-known "Problem of Priors" in Bayesian analysis, by reversing the conventional priorlikelihood-posterior ("forward") use of Bayes' theorem. Such Reverse-Bayes analysis allows priors to be deduced from the likelihood by requiring that the posterior achieve a specified level of credibility. We summarise the technical underpinning of this approach, and show how it opens up new approaches to common inferential challenges, such as assessing the credibility of scientific findings, setting them in appropriate context, estimating the probability of successful replications, and extracting more insight from NHST while reducing the risk of misinterpretation. We argue that Reverse-Bayes methods have a key role to play in making Bayesian methods more accessible and attractive for evidence assessment and research synthesis. As a running example we consider a recently published meta-analysis from several randomised controlled trials (RCTs) investigating the association between corticosteroids and mortality in hospitalised patients with COVID-19.

show abstract

Section: Discussionmentioning

confidence: 99%

“… 24 , 25 , 26 The appeal of Reverse‐Bayes techniques has recently been widened by the development of inferential methods using both posterior probabilities and Bayes factors. 18 , 29 …”

Section: Discussionmentioning

confidence: 99%

“…We could again use the Box 11 prior‐predictive check as in Section 2.2, however, the resulting tail probability is difficult to compare to the Bayes‐factor cut‐off

γ .

θ ∣ H_{2} \sim normalN ((),, \overset{truê}{θ}, σ^{2})

the posterior of

θ

based on the data and the reference prior

f ((), θ) \propto 1

. The optimistic prior therefore represents the position of a proponent who takes the original claim at face value.…”

Section: Reverse‐bayes Methods With Bayes Factorsmentioning

confidence: 99%

Section: Introduction: the Origin Of Reverse‐bayes Methodsmentioning

confidence: 99%

“…Intrinsic credibility at level

is established when,

and we are interested in the Bayes factor for intrinsic credibility

which is the smallest level

where ( A1 ) holds. The

is therefore a special case of the sceptical Bayes factor from Pawel and Held 29 where the same data is used in both Bayes factors (instead of the data from a replication study for

It is hence given by,

with

, and

the branch of the Lambert

function that satisfies