Bayes Factor Model Comparisons Across Parameter Values for Mixed Models

Abstract

Nested data structures, in which conditions include multiple trials, are often analyzed using repeated-measures analysis of variance or mixed effects models. Typically, researchers are interested in determining whether there is an effect of the experimental manipulation. Unfortunately, these kinds of analyses have different appropriate specifications for the null and alternative models, and a discussion on which is to be preferred and when is sorely lacking. van Doorn et al. (2021) performed three types of Bayes factor model comparisons on a simulated data set in order to examine which model comparison is most suitable for quantifying evidence for or against the presence of an effect of the experimental manipulation. Here we extend their results by simulating multiple data sets for various scenarios and by using different prior specifications. We demonstrate how three different Bayes factor model comparison types behave under changes in different parameters, and we make concrete recommendations on which model comparison is most appropriate for different scenarios.

Decisions about equivalence: A comparison of TOST, HDI-ROPE, and the Bayes factor.

“…As a result, we recommend the use of Bayes factors for quantifying evidence toward equivalence. Bayes factors for equivalence designs can easily be calculated with the baymedr (Linde & van Ravenzwaaij, 2019b) or the BayesFactor (Morey & Rouder, 2018) packages written in R (R Core Team, 2019) or with JASP (JASP Team, 2020).…”

“…The calculation of the boundaries of the 95% HDI of the posterior was achieved using functions developed by Gronau et al (2020; available at https://osf.io/bsp6z/), which are available for R (R Core Team, 2019). Bayes factors were obtained using the baymedr software (Linde & van Ravenzwaaij, 2019b) written in R (R Core Team, 2019). These Bayes factors were compared to two thresholds, BF thr = {3, 10}, which follow approximate thresholds for judging the magnitude of the evidence: 1 < BF < 3 for “anecdotal evidence”; 3 < BF < 10 for “moderate evidence”; 10 < BF < 30 for “strong evidence”; 30 < BF < 100 for “very strong evidence”; BF > 100 for “extreme evidence” (cf.…”

A review of applications of the Bayes factor in psychological research.

“…The prior for the (standardized) population effect size δ under the alternative is a truncated Cauchy(0, 1/2) distribution with the two values denoting the location and scale parameters, respectively 7 . The analysis was carried out using the baymedr package (Linde & van Ravenzwaaij, 2021).…”

“…On the OSF website corresponding to this article, additional examples illustrate the evaluation of interval hypotheses for equivalence testing with complementary hypotheses and for the application of noninferiority testing in the context of financial statement audits. The main software resources for the evaluation of interval hypotheses are the R packages baymedr (Linde & van Ravenzwaaij, 2021), BayesFactor (Morey & Rouder, 2018), bain (Gu et al, 2019; partly also implemented in JASP which offers an easy-to-use GUI; JASP Team, 2020), and BFpack (Mulder et al, 2019).…”