How to quantify support for and against the null hypothesis: A flexible WinBUGS implementation of a default Bayesian t test

Wetzels, Ruud; Raaijmakers, Jeroen G. W.; Jakab, Emőke; Wagenmakers, Eric‐Jan

doi:10.3758/pbr.16.4.752

Cited by 136 publications

(167 citation statements)

References 47 publications

Supporting

Mentioning

164

Contrasting

Unclassified

Order By: Relevance

“…Bayesian hypothesis testing allows one to obtain evidence in favor of the null hypothesis. Because theories and models often predict the absence of a difference, it is vital for scientific progress to be able to quantify evidence in favor of the null hypothesis (e.g., Gallistel, 2009;Rouder et al, 2009;Wetzels et al, 2009). In the field of visual word recognition, for instance, the entry-opening theory (Forster, Mohan, & Hector, 2003) predicts that masked priming is absent for items that do not have a lexical representation;…”

Section: Additional Advantages Of Bayesian Hypothesis Testingmentioning

confidence: 99%

“…To the best of our knowledge, the Savage-Dickey method has only been used in psychology once before, by Wetzels et al (2009), who used it to develop a WinBUGS implementation of the t-test proposed by Rouder et al (2009).…”

Section: The Savage-dickey Density Ratiomentioning

confidence: 99%

“…Second, one can explore Bayesian statistical procedures that provide exactly what the Id wants-probabilities for hypotheses. Using Bayesian procedures, one can quantify support both in favor of and against the null hypothesis (Gallistel, 2009;Rouder, Speckman, Sun, Morey, & Iverson, 2009;Wetzels, Raaijmakers, Jakab, & Wagenmakers, 2009), and one can state that the probability that a parameter l lies in a 95% ''credible interval" is, indeed, .95. In this article, we promote the second course of action.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Bayesian hypothesis testing for psychologists: A tutorial on the Savage–Dickey method

Wagenmakers

Lodewyckx

Kuriyal

et al. 2010

Cognitive Psychology

Self Cite

652

642

View full text Add to dashboard Cite

a b s t r a c tIn the field of cognitive psychology, the p-value hypothesis test has established a stranglehold on statistical reporting. This is unfortunate, as the p-value provides at best a rough estimate of the evidence that the data provide for the presence of an experimental effect. An alternative and arguably more appropriate measure of evidence is conveyed by a Bayesian hypothesis test, which prefers the model with the highest average likelihood. One of the main problems with this Bayesian hypothesis test, however, is that it often requires relatively sophisticated numerical methods for its computation. Here we draw attention to the Savage-Dickey density ratio method, a method that can be used to compute the result of a Bayesian hypothesis test for nested models and under certain plausible restrictions on the parameter priors. Practical examples demonstrate the method's validity, generality, and flexibility.

show abstract

Section: Additional Advantages Of Bayesian Hypothesis Testingmentioning

confidence: 99%

Section: The Savage-dickey Density Ratiomentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Bayesian hypothesis testing for psychologists: A tutorial on the Savage–Dickey method

Wagenmakers

Lodewyckx

Kuriyal

et al. 2010

Cognitive Psychology

Self Cite

652

642

View full text Add to dashboard Cite

show abstract

“…The standard deviations, 1 and 2 , each have folded Cauchy ϩ (0,1) priors instead of the gamma(0.5,0.5) distribution used by Rouder et al In the alternative hypothesis, the effect size ␦ has a Cauchy(0,1) prior, the same as used by Rouder et al The group means are 1 ϭ ϩ ␣/2 and 2 ϭ Ϫ ␣/2, where ␣ ϭ ␦ pooled and pooled ϭ ͱ͑ 1 2 ͑N 1 Ϫ 1͒ ϩ 2 2 ͑N 2 Ϫ 1͒/͑N 1 ϩ N 2 Ϫ 2͒ (Hedges, 1981). Instead of deriving the BF analytically, Wetzels et al (2009) used MCMC methods to obtain a posterior distribution on the parameters in the alternative model and then adapted the SavageDickey (SD) method to approximate the Bayes factor (Dickey & Lientz, 1970). The SD method assumes that the prior on the variance in the alternative model, at the null value, equals the prior on the variance in the null model: p( 2 ͉H alt , ␦ ϭ 0) ϭ p( 2 ͉H null ).…”

Section: Mcmc Approach To Bayes Factormentioning

confidence: 99%

“…In principle, many different data sets could have generated the posterior parameter distribution, and therefore the data should not be used in re-describing the posterior. Nevertheless, some users may prefer to compute an effect size in which the estimates are weighted by the sample sizes in the groups: (Hedges, 1981;Wetzels et al, 2009). This form does not change the sign of the effect size, merely its magnitude, so the proportion of the posterior distribution of the effect size that is greater (or less) than zero remains unaffected.…”

Section: Power Analysis For Bayesian Estimationmentioning

confidence: 99%

Bayesian Estimation Supersedes the t Test

Kruschke¹

2012

PsycEXTRA Dataset

316

412

View full text Add to dashboard Cite

Bayesian estimation for 2 groups provides complete distributions of credible values for the effect size, group means and their difference, standard deviations and their difference, and the normality of the data. The method handles outliers. The decision rule can accept the null value (unlike traditional t tests) when certainty in the estimate is high (unlike Bayesian model comparison using Bayes factors). The method also yields precise estimates of statistical power for various research goals. The software and programs are free and run on Macintosh, Windows, and Linux platforms.Keywords: Bayesian statistics, effect size, robust estimation, Bayes factor, confidence interval One of the most frequently encountered scientific procedures is a comparison of two groups (e.g., du Prel, Röhrig, Hommel, & Blettner, 2010;Fritz, Morris, & Richler, 2012;Wetzels et al., 2011). Given data from two groups, researchers ask various comparative questions: How much is one group different from another? Can we be reasonably sure that the difference is non-zero? How certain are we about the magnitude of difference? These questions are difficult to answer because data are contaminated by random variability despite researchers' efforts to minimize extraneous influences on the data. Because of "noise" in the data, researchers rely on statistical methods of probabilistic inference to interpret the data. When data are interpreted in terms of meaningful parameters in a mathematical description, such as the difference of mean parameters in two groups, it is Bayesian analysis that provides complete information about the credible parameter values. Bayesian analysis is also more intuitive than traditional methods of null hypothesis significance testing (e.g., Dienes, 2011).This article introduces an intuitive Bayesian approach to the analysis of data from two groups. The method yields complete distributional information about the means and standard deviations of the groups. In particular, the analysis reveals the relative credibility of every possible difference of means, every possible difference of standard deviations, and all possible effect sizes. From this explicit distribution of credible parameter values, inferences about null values can be made without ever referring to p values as in null hypothesis significance testing (NHST). Unlike NHST, the Bayesian method can accept the null value, not only reject it, when certainty in the estimate is high. The new method handles outliers by describing the data as heavy tailed distributions instead of normal distributions, to the extent implied by the data. The new method also implements power analysis in both retrospective and prospective forms.The analysis is implemented in the widely used and free programming languages R and JAGS and can be run on Macintosh, Linux, and Windows operating systems. Complete installation instructions are provided, along with working examples. The programs can also be flexibly extended to other types of data and analyses. Thus, the software can be used by virtually ...

show abstract

Enhancing computational thinking in undergraduate engineering courses using model‐eliciting activities

Liu

Jian-wen

2020

Comp Applic In Engineering

View full text Add to dashboard Cite

Fostering of computational thinking (CT) skills has become an essential goal for engineering courses. This paper provides and discusses a teaching approach based on model‐eliciting activity (MEA) for enhancing students' CT in existing engineering courses. To verify the proposed approach, experimental studies were performed with students in the practicum unit of a computer‐aided engineering course. First, we investigated the feasibility of integrating CT training into the engineering courses through guiding students to perform a topic‐specific design task. Second, we compared the influences of the MEA‐based approach and conventional lecture‐based approach on students' learning effect with respect to course objectives and CT skills. Results show the CT‐integrated teaching did not at least weaken the teaching effect of the engineering course, and the MEA‐based approach seems to be more conducive to strengthening students' skills of identifying and solving problems, as well as team cooperation. Results also show the learning effect of the MEA group has apparent advantages for enhancing CT skills. These findings suggest that using MEA in engineering courses is beneficial to promote students' CT skills in terms of simplification, embedding, transformation, and simulation.

show abstract

How to quantify support for and against the null hypothesis: A flexible WinBUGS implementation of a default Bayesian t test

Cited by 136 publications

References 47 publications

Bayesian hypothesis testing for psychologists: A tutorial on the Savage–Dickey method

Bayesian hypothesis testing for psychologists: A tutorial on the Savage–Dickey method

Bayesian Estimation Supersedes the t Test

Enhancing computational thinking in undergraduate engineering courses using model‐eliciting activities

Contact Info

Product

Resources

About