Estimates of relative acceptability from paired preference tests

Abstract

The relative acceptability of a food or beverage is an important data point in many different scenarios. In humans, a nine-point category scale is typically used when the hedonic characteristics of several products are of interest. However, these scales are not consistently used by participants and therefore reliability is low. This article outlines the effectiveness of four techniques (mElo, Thurstonian modeling, Bradley-Terry, and Friedman) to calculate numerical values for products based upon their performance in a paired preference paradigm. In this study, acceptance data from four separate studies were compared to numeric scores constructed from a paired preference paradigm. In general, numeric ratings constructed from paired comparisons correlated very well with the mean overall liking ratings. The relationship between these acceptance ratings and numerical preference ratings did show to be somewhat dependent on the type of food product. This study serves as a guide to those looking to further quantify paired preference data. Practical Applications: Paired preference tests are simple and easy to understand for those who struggle with test instructions and numerical scales. However, in a multisample paradigm, sensory scientists have overwhelmingly chosen to use rating scales for a variety of reasons. This study shows the effectiveness of alternative data analysis techniques to extract numerical values for relative acceptance based on paired preference tests. This information could be used in the implementation of paired preference tests to replace traditional affective testing in groups that do not perform well with scales (e.g., children) or as an alternate analysis method for paired preference tests.

Bayesian Paired-Comparison with the bpcs Package

“…To exemplify the basic usage of the bpcs package, Luckett et al (2020) work is used as an example.The authors investigate the relative acceptability of food and beverage choices using paired preferences. One of the examples discussed is the acceptability of five brands of four cheese frozen pizzas.…”

“…When it comes to the respondents' perspective, there is vast documentation of difficulties that participants may experience when rating perceptions with scales. They may struggle to use the scale correctly, they may try to self-monitor their answers, and for specific samples, the scales may not even be effective (such as young children, people with low literacy, or when respondents are using their second language to answer the scales) (Coetzee & Taylor, 1996;Luckett, Burns, & Jenkinson, 2020;Petrou, 2003).…”

“…In the context of this article and the context of applied psychological and behavioral research, the stimulus-centric approach is the one that fits better, and it is in this context that the Bradley-Terry model could be most useful. Examples are the preferences of natural landscapes (Hägerhäll et al, 2018), preferences for stimulus (Chien et al, 2012), for food (Coetzee & Taylor, 1996;Luckett et al, 2020), analysis of animal dominance (Abalos, i de Lanuza, Carazo, & Font, 2016;Bush, Quinn, Balreira, & Johnson, 2016;Miller et al, 2017) and in the use of pharmacological medication (Meid et al, 2016).…”

Bayesian paired comparison with the bpcs package

“…or "DiGiorno or Freschetta?" (Luckett, Burns, & Jenkinson, 2020). The most common modeling technique for paired comparisons and the focus of this article is the Bradley-Terry model and its extensions (Bradley & Terry, 1952).…”

“…Ordinal scales can also be used to assess preferences, but they may lead to several difficulties. For example, participants may struggle to use the scale correctly (Coetzee & Taylor, 1996;Petrou, 2003), they may try to self-monitor their answers (Kreitchmann, Abad, Ponsoda, Nieto, & Morillo, 2019;Hontangas et al, 2015), and for specific samples, the scales may not even be effective, such as in animal behavior studies, studies with young children, people with low literacy, or when respondents are using their second language to answer the scales (Luckett et al, 2020;Hopper, Egelkamp, Fidino, & Ross, 2019;Huskisson, Jacobson, Egelkamp, Ross, & Hopper, 2020).…”