The Weighted Likelihood Ratio, Sharp Hypotheses about Chances, the Order of a Markov Chain

a b s t r a c tIn the field of cognitive psychology, the p-value hypothesis test has established a stranglehold on statistical reporting. This is unfortunate, as the p-value provides at best a rough estimate of the evidence that the data provide for the presence of an experimental effect. An alternative and arguably more appropriate measure of evidence is conveyed by a Bayesian hypothesis test, which prefers the model with the highest average likelihood. One of the main problems with this Bayesian hypothesis test, however, is that it often requires relatively sophisticated numerical methods for its computation. Here we draw attention to the Savage-Dickey density ratio method, a method that can be used to compute the result of a Bayesian hypothesis test for nested models and under certain plausible restrictions on the parameter priors. Practical examples demonstrate the method's validity, generality, and flexibility.

show abstract

“…This surprising result was first published by Dickey and Lientz (1970), who attributed it to Leonard J. ''Jimmie" Savage.…”

Section: The Savage-dickey Density Ratiomentioning

confidence: 90%

“…This appendix provides the derivation of the Savage-Dickey density ratio (e.g., Dickey & Lientz, 1970, Lindley, 1972, pp. 30-32, O'Hagan & Forster, 2004.…”

Section: Appendix a Derivation Of The Savage-dickey Density Ratiomentioning

confidence: 99%

Bayesian hypothesis testing for psychologists: A tutorial on the Savage–Dickey method

Wagenmakers

Lodewyckx

Kuriyal

et al. 2010

Cognitive Psychology

652

642

View full text Add to dashboard Cite

show abstract

“…The standard deviations, 1 and 2 , each have folded Cauchy ϩ (0,1) priors instead of the gamma(0.5,0.5) distribution used by Rouder et al In the alternative hypothesis, the effect size ␦ has a Cauchy(0,1) prior, the same as used by Rouder et al The group means are 1 ϭ ϩ ␣/2 and 2 ϭ Ϫ ␣/2, where ␣ ϭ ␦ pooled and pooled ϭ ͱ͑ 1 2 ͑N 1 Ϫ 1͒ ϩ 2 2 ͑N 2 Ϫ 1͒/͑N 1 ϩ N 2 Ϫ 2͒ (Hedges, 1981). Instead of deriving the BF analytically, Wetzels et al (2009) used MCMC methods to obtain a posterior distribution on the parameters in the alternative model and then adapted the SavageDickey (SD) method to approximate the Bayes factor (Dickey & Lientz, 1970). The SD method assumes that the prior on the variance in the alternative model, at the null value, equals the prior on the variance in the null model: p( 2 ͉H alt , ␦ ϭ 0) ϭ p( 2 ͉H null ).…”

Section: Mcmc Approach To Bayes Factormentioning

confidence: 99%

Bayesian Estimation Supersedes the t Test

Kruschke¹

2012

PsycEXTRA Dataset

316

412

View full text Add to dashboard Cite

Bayesian estimation for 2 groups provides complete distributions of credible values for the effect size, group means and their difference, standard deviations and their difference, and the normality of the data. The method handles outliers. The decision rule can accept the null value (unlike traditional t tests) when certainty in the estimate is high (unlike Bayesian model comparison using Bayes factors). The method also yields precise estimates of statistical power for various research goals. The software and programs are free and run on Macintosh, Windows, and Linux platforms.Keywords: Bayesian statistics, effect size, robust estimation, Bayes factor, confidence interval One of the most frequently encountered scientific procedures is a comparison of two groups (e.g., du Prel, Röhrig, Hommel, & Blettner, 2010;Fritz, Morris, & Richler, 2012;Wetzels et al., 2011). Given data from two groups, researchers ask various comparative questions: How much is one group different from another? Can we be reasonably sure that the difference is non-zero? How certain are we about the magnitude of difference? These questions are difficult to answer because data are contaminated by random variability despite researchers' efforts to minimize extraneous influences on the data. Because of "noise" in the data, researchers rely on statistical methods of probabilistic inference to interpret the data. When data are interpreted in terms of meaningful parameters in a mathematical description, such as the difference of mean parameters in two groups, it is Bayesian analysis that provides complete information about the credible parameter values. Bayesian analysis is also more intuitive than traditional methods of null hypothesis significance testing (e.g., Dienes, 2011).This article introduces an intuitive Bayesian approach to the analysis of data from two groups. The method yields complete distributional information about the means and standard deviations of the groups. In particular, the analysis reveals the relative credibility of every possible difference of means, every possible difference of standard deviations, and all possible effect sizes. From this explicit distribution of credible parameter values, inferences about null values can be made without ever referring to p values as in null hypothesis significance testing (NHST). Unlike NHST, the Bayesian method can accept the null value, not only reject it, when certainty in the estimate is high. The new method handles outliers by describing the data as heavy tailed distributions instead of normal distributions, to the extent implied by the data. The new method also implements power analysis in both retrospective and prospective forms.The analysis is implemented in the widely used and free programming languages R and JAGS and can be run on Macintosh, Linux, and Windows operating systems. Complete installation instructions are provided, along with working examples. The programs can also be flexibly extended to other types of data and analyses. Thus, the software can be used by virtually ...

show abstract

“…To obtain the Bayes factor, we use a method that is simple, intuitive, and flexible: the SD density ratio method (e.g., Dickey & Lientz, 1970;O'Hagan & Forster, 2004, pp. 174-177;Verdinelli & Wasserman, 1995).…”

Section: Bf Ng T V T Ngmentioning

confidence: 99%

How to quantify support for and against the null hypothesis: A flexible WinBUGS implementation of a default Bayesian t test

Wetzels

Raaijmakers

Jakab

et al. 2009

Psychonomic Bulletin & Review

136

158

View full text Add to dashboard Cite

show abstract

The Weighted Likelihood Ratio, Sharp Hypotheses about Chances, the Order of a Markov Chain

Cited by 233 publications

References 0 publications

Bayesian hypothesis testing for psychologists: A tutorial on the Savage–Dickey method

Bayesian hypothesis testing for psychologists: A tutorial on the Savage–Dickey method

Bayesian Estimation Supersedes the t Test

How to quantify support for and against the null hypothesis: A flexible WinBUGS implementation of a default Bayesian t test

Contact Info

Product

Resources

About