Bayesian Paired-Comparison with the bpcs Package

Abstract

This article introduces the bpcs R package (Bayesian Paired Comparison in Stan) and the statistical models implemented in the package. The goal of this package is to facilitate the use of Bayesian models for paired comparison data in behavioral research. Historically, studies on preferences have relied on Likert scale assessments and the frequentist approach to analyze the data. As an alternative, this article proposes the use of Bayesian models for forced choices assessments. The advantages of forced-choice assessments are that they minimize common bias that Likert scales are susceptible to, they can increase criterion validity, control for faking responses, and are quite useful to assess preferences in studies with children and non-humans primates. Bayesian data analyses are less sensitive to optional stopping, have better control of type I error, have stronger evidence towards the null hypothesis, allows propagation of uncertainties, includes prior information, and perform well when handling models with many parameters and latent variables. The bpcs package provides a consistent interface for R users and several functions to evaluate the posterior distribution of all parameters, to estimate the posterior distribution of any contest between items, and to obtain the posterior distribution of the ranks. Three reanalyses of recent studies that used the frequentist Bradley-Terry model are presented. These reanalyses are conducted with the Bayesian models of the bpcs package and all the code used to fit the models, generate the figures and the tables are available in the online appendix.

Roundtrip, free-floating and peer-to-peer carsharing: A Bayesian behavioral analysis

“…This variance was set to be weakly-informative, i.e. to reduce the influence of the prior in the model convergence, while providing some level of regularisation to the model (Mattos and Ramos, 2021). This prior allows probabilities to be of i beating j to be in the range of 0.0001 to 0.9998.…”

“…The data was analysed using the statistical software R version 4.0.3. The statistical models were developed using the brms package for Bayesian regression modelling (Bürkner, 2018), including the multinomial and cumulative ordered logit regression; the bpcs package for the Bayesian Bradley-Terry model (Mattos and Ramos, 2021); and the psych package for factorial analysis (Revelle and Revelle, 2015).…”