A Critical Evaluation of the FBST ev for Bayesian Hypothesis Testing

2021

Comput Brain Behav

The Full Bayesian Significance Test (FBST) and the Bayesian evidence value recently have received increasing attention across a variety of sciences including psychology. Ly and Wagenmakers (2021) have provided a critical evaluation of the method and concluded that it suffers from four problems which are mostly attributed to the asymptotic relationship of the Bayesian evidence value to the frequentist p-value. While Ly and Wagenmakers (2021) tackle an important question about the best way of statistical hypothesis testing in the cognitive sciences, it is shown in this paper that their arguments are based on a specific measure-theoretic premise. The identified problems hold only under a specific class of prior distributions which are required only when adopting a Bayes factor test. However, the FBST explicitly avoids this premise, which resolves the problems in practical data analysis. In summary, the analysis leads to the more important question whether precise point null hypotheses are realistic for scientific research, and a shift towards the Hodges-Lehmann paradigm may be an appealing solution when there is doubt on the appropriateness of a precise hypothesis.

Section: Measure-theoretic Background-the Fbstmentioning

confidence: 99%

Section: Measure-theoretic Background-the Fbstmentioning

confidence: 99%

Section: Measure-theoretic Background-the Fbstmentioning

confidence: 99%

Section: Measure-theoretic Background-the Fbstmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

On the Measure-Theoretic Premises of Bayes Factor and Full Bayesian Significance Tests: a Critical Reevaluation

2021

Comput Brain Behav

The evidence interval and the Bayesian evidence value: On a unified theory for Bayesian hypothesis testing and interval estimation

2022

Brit J Math & Statis

Interval estimation is one of the most frequently used methods in statistical science, employed to provide a range of credible values a parameter is located in after taking into account the uncertainty in the data. However, while this interpretation only holds for Bayesian interval estimates, these suffer from two problems. First, Bayesian interval estimates can include values which have not been corroborated by observing the data. Second, Bayesian interval estimates and hypothesis tests can yield contradictory conclusions. In this paper a new theory for Bayesian hypothesis testing and interval estimation is presented. A new interval estimate is proposed, the Bayesian evidence interval, which is inspired by the Pereira–Stern theory of the full Bayesian significance test (FBST). It is shown that the evidence interval is a generalization of existing Bayesian interval estimates, that it solves the problems of standard Bayesian interval estimates and that it unifies Bayesian hypothesis testing and parameter estimation. The Bayesian evidence value is introduced, which quantifies the evidence for the (interval) null and alternative hypothesis. Based on the evidence interval and the evidence value, the (full) Bayesian evidence test (FBET) is proposed as a new, model‐independent Bayesian hypothesis test. Additionally, a decision rule for hypothesis testing is derived which shows the relationship to a widely used decision rule based on the region of practical equivalence and Bayesian highest posterior density intervals and to the e‐value in the FBST. In summary, the proposed method is universally applicable, computationally efficient, and while the evidence interval can be seen as an extension of existing Bayesian interval estimates, the FBET is a generalization of the FBST and contains it as a special case. Together, the theory developed provides a unification of Bayesian hypothesis testing and interval estimation and is made available in the R package fbst.

On Two Measure-Theoretic Aspects of the Full Bayesian Significance Test for Precise Bayesian Hypothesis Testing †

The 40th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering

2021

The Full Bayesian Significance Test (FBST) has been proposed as a convenient method to replace frequentist p-values for testing a precise hypothesis. Although the FBST enjoys various appealing properties, the purpose of this paper is to investigate two aspects of the FBST which are sometimes observed as measure-theoretic inconsistencies of the procedure and have not been discussed rigorously in the literature. First, the FBST uses the posterior density as a reference for judging the Bayesian statistical evidence against a precise hypothesis. However, under absolutely continuous prior distributions, the posterior density is defined only up to Lebesgue null sets which renders the reference criterion arbitrary. Second, the FBST statistical evidence seems to have no valid prior probability. It is shown that the former aspect can be circumvented by fixing a version of the posterior density before using the FBST, and the latter aspect is based on its measure-theoretic premises. An illustrative example demonstrates the two aspects and their solution. Together, the results in this paper show that both of the two aspects which are sometimes observed as measure-theoretic inconsistencies of the FBST are not tenable. The FBST thus provides a measure-theoretically coherent Bayesian alternative for testing a precise hypothesis.