The Bayesian New Statistics: Hypothesis testing, estimation, meta-analysis, and power analysis from a Bayesian perspective

Abstract

In the practice of data analysis, there is a conceptual distinction between hypothesis testing, on the one hand, and estimation with quantified uncertainty on the other. Among frequentists in psychology, a shift of emphasis from hypothesis testing to estimation has been dubbed "the New Statistics" (Cumming, 2014). A second conceptual distinction is between frequentist methods and Bayesian methods. Our main goal in this article is to explain how Bayesian methods achieve the goals of the New Statistics better than frequentist methods. The article reviews frequentist and Bayesian approaches to hypothesis testing and to estimation with confidence or credible intervals. The article also describes Bayesian approaches to meta-analysis, randomized controlled trials, and power analysis.Keywords Null hypothesis significance testing · Bayesian inference · Bayes factor · Confidence interval · Credible interval · Highest density interval · Region of practical equivalence · Meta-analysis · Power analysis · Effect size · Randomized controlled trial · Equivalence testingThe New Statistics emphasizes a shift of emphasis away from null hypothesis significance testing (NHST) to "estimation based on effect sizes, confidence intervals, and metaanalysis" (Cumming, 2014, p. 7). There are many reasons John K. Kruschke johnkruschke@gmail.com 1 Indiana University, Bloomington, USA to eschew NHST, with its seductive lapse to black-and-white thinking about the presence or absence of effects. There are also many reasons to promote instead a cumulative science that incrementally improves estimates of magnitudes and uncertainty. These reasons were recently highlighted in a prominent statement from the American Statistical Association (ASA; Wasserstein & Lazar, 2016) that will be summarized later in this article. Recent decades have also seen repeated calls to shift emphasis away from frequentist methods to Bayesian analysis (e.g., Lindley, 1975).In this article, we review both of these recommended shifts of emphasis in the practice of data analysis, and we promote their convergence in Bayesian methods for estimation. The goals of the New Statistics are better achieved by Bayesian methods than by frequentist methods. In that sense, we recommend a Bayesian New Statistics. Within the domain of Bayesian methods, we have a more nuanced emphasis. Bayesian methods provide a coherent framework for hypothesis testing, so when null hypothesis testing is the crux of the research then Bayesian null hypothesis testing should be carefully used. But we also believe that typical analyses should not routinely stop with hypothesis testing alone. In that sense, we recommend a New Bayesian Statistics, that is, Bayesian analyses that also consider estimates of magnitudes and uncertainty, along with meta-analyses.This article begins with an extensive description of frequentist and Bayesian approaches to null hypothesis testing and estimation with confidence or credible intervals. Subsequently, the article explains Bayesian approaches to metaanalysis, rando...

Bayesian data analysis for newcomers

“…The emphases are on establishing foundational concepts and on disabusing misconceptions. This article does not rehearse the many reasons to be wary of p values and confidence intervals (see, for example, Kruschke, 2013;Kruschke & Liddell, 2017;Wagenmakers, 2007). We assume that you already are curious about Bayesian methods and you want to learn about them but you have little previous exposure to them.…”

Why do we need to employ Bayesian statistics and how can we employ it in studies of moral education?: With practical guidelines to use JASP for educators and researchers

“…), which might seem to be less straightforward to interpret among non-experts compared with p-values, to make a better judgment from the frequentist perspective. In addition, we still cannot have any direct information about whether the collected data supports our hypotheses with the reported RUNNING HEAD: UTILIZING BAYESIAN STATISTICS 19 frequentist indicators; such indicators inform us whether it is possible to reject null hypotheses (Kruschke & Liddell, 2018). If we utilize Bayesian techniques, we simple need to interpret one indicator, a Bayes factor, based on the suggested threshold values (e.g., Kass & Raftery, 1995).…”

No Need for Bayes Factors: A Fully Bayesian Evidence Synthesis

“…However, it would be naïve to think that because the two approaches provide a similar conclusion, it matters little what approach researchers use for synthesizing evidence in a Bayesian fashion (see [4] for a similar warning). Accordingly, we provide a free online "Fully Bayesian Evidence Synthesis" application that allows scholars to implement the proposed estimation approach and download the density distributions and simulated data.…”

“…In addition to their ambiguity, Bayes factors provide an incoherent framework for evidential support because of their high sensitivity to the prior and because they penalize hypotheses that contain values with low likelihoods [3,4,8]. In short, while there are many "Bayesian" approaches, Bayes factors in particular "have no direct foundational meaning to a Bayesian", as only posterior probabilities have a proper Bayesian interpretation ( [8], p. 56).…”

“…Indeed, although Bayes Factors are now advocated widely in psychology in place of pvalues and null hypothesis significance testing (NHST), Bayes factors suffer from some of the same fundamental flaws [3], and much like the "old" statistics, they do not reveal useful information about magnitude and uncertainty [4,5]. Moreover, although simple rules-of-thumb exist (e.g., see [6]), what constitutes reliable evidence in favor of the null hypothesis is largely unclear and ambiguous.…”