Bayesian hypothesis testing presents an attractive alternative to p value hypothesis testing. Part I of this series outlined several advantages of Bayesian hypothesis testing, including the ability to quantify evidence and the ability to monitor and update this evidence as data come in, without the need to know the intention with which the data were collected. Despite these and other practical advantages, Bayesian hypothesis tests are still reported relatively rarely. An important impediment to the widespread adoption of Bayesian tests is arguably the lack of user-friendly software for the run-of-the-mill statistical problems that confront psychologists for the analysis of almost every experiment: the t-test, ANOVA, correlation, regression, and contingency tables. In Part II of this series we introduce JASP (http://www.jasp-stats.org), an open-source, cross-platform, user-friendly graphical software package that allows users to carry out Bayesian hypothesis tests for standard statistical problems. JASP is based in part on the Bayesian analyses implemented in Morey and Rouder’s BayesFactor package for R. Armed with JASP, the practical advantages of Bayesian hypothesis testing are only a mouse click away.

Many psychologists do not realize that exploratory use of the popular multiway analysis of variance harbors a multiple-comparison problem. In the case of two factors, three separate null hypotheses are subject to test (i.e., two main effects and one interaction). Consequently, the probability of at least one Type I error (if all null hypotheses are true) is 14 % rather than 5 %, if the three tests are independent. We explain the multiple-comparison problem and demonstrate that researchers almost never correct for it. To mitigate the problem, we describe four remedies: the omnibus F test, control of the familywise error rate, control of the false discovery rate, and preregistration of the hypotheses.

The Iowa Gambling Task (IGT; Bechara, Damasio, Damasio, & Anderson, 1994) is often used to assess decision-making deficits in clinical populations. The interpretation of the results hinges on 3 key assumptions: (a) healthy participants learn to prefer the good options over the bad options; (b) healthy participants show homogeneous choice behavior; and (c) healthy participants first explore the different options and then exploit the most profitable ones. Here we test these assumptions using 2 extensive literature reviews and analysis of 8 data sets. The results show that all 3 assumptions may be invalid; that is, (a) healthy participants often prefer decks with infrequent losses; (b) healthy participants show idiosyncratic choice behavior; and (c) healthy participants do not show a systematic decrease in the number of switches across trials. Our findings question the prevailing interpretation of IGT data and suggest that, in future applications of the IGT, key assumptions about performance of healthy participants warrant close scrutiny.

The marginal likelihood plays an important role in many areas of Bayesian statistics such as parameter estimation, model comparison, and model averaging. In most applications, however, the marginal likelihood is not analytically tractable and must be approximated using numerical methods. Here we provide a tutorial on bridge sampling (Bennett, 1976; Meng & Wong, 1996), a reliable and relatively straightforward sampling method that allows researchers to obtain the marginal likelihood for models of varying complexity. First, we introduce bridge sampling and three related sampling methods using the beta-binomial model as a running example. We then apply bridge sampling to estimate the marginal likelihood for the Expectancy Valence (EV) model—a popular model for reinforcement learning. Our results indicate that bridge sampling provides accurate estimates for both a single participant and a hierarchical version of the EV model. We conclude that bridge sampling is an attractive method for mathematical psychologists who typically aim to approximate the marginal likelihood for a limited set of possibly high-dimensional models.

Decision-making deficits in clinical populations are often studied using the Iowa Gambling Task (IGT). Performance on the IGT can be decomposed in its constituent psychological processes by means of cognitive modeling analyses. However, conclusions about the hypothesized psychological processes are valid only if the model provides an adequate account of the data. In this article, we systematically assessed absolute model performance of the Expectancy Valence (EV) model, the Prospect Valence Learning (PVL) model, and a hybrid version of both models-the PVL-Delta model-using 2 different methods. These methods assess (a) whether a model provides an acceptable fit to an observed choice pattern, and (b) whether the parameters obtained from model fitting can be used to generate the observed choice pattern. Our results show that all models provided an acceptable fit to 2 stylized data sets; however, when the model parameters were used to generate choices, only the PVL-Delta model captured the qualitative patterns in the data. These findings were confirmed by fitting the models to 5 published IGT data sets. Our results highlight that a model's ability to fit a particular choice pattern does not guarantee that the model can also generate that same choice pattern. Future applications of RL models should carefully assess absolute model performance to avoid premature conclusions about the psychological processes that drive performance on the IGT.