Focusing on what matters: Restructuring the presentation of Bayesian reasoning problems.

Abstract

We propose that a mismatch in problem presentation and question structures may promote errors on Bayesian reasoning problems. In this task, people determine the likelihood that a positive test actually indicates the presence of a condition. Research has shown that people routinely fail to correctly identify this positive predictive value (PPV). We point out that the typical problem structure is likely to confuse reasoners by focusing on the incorrect reference class for answering this diagnostic question; instead, providing the anchor needed to address the diagnostic question about sensitivity (SEN). Results of two experiments are described in which participants answer diagnostic questions using problems presented with congruent or incongruent reference classes. Aligning reference classes eases both representational and computational difficulties, increasing the proportion who were consistently accurate to an unprecedented 93% on PPV questions, and 69% on SEN questions. Analysis of response components from incongruent problems indicated that many errors reflect difficulties in identifying and applying appropriate values from the problem, which are prerequisite processes that contribute to computational errors. We conclude with a discussion of the need, especially in applied settings and on initial exposure, to adopt problem presentations to guide, rather than confuse, the organization and use of diagnostic information.

Reference Dependence in Bayesian Reasoning: Value Selection Bias, Congruence Effects, and Response Prompt Sensitivity

“…These types of tests are commonly presented as Bayesian reasoning problems, which are used to evaluate the ability to update prior beliefs based on additional evidence to determine a posterior probability. Research over the last 40 years demonstrates that uninitiated or novice reasoners tend to have difficulty determining the PPV (e.g., Gigerenzer and Hoffrage, 1995 ; Gigerenzer et al, 2007 ; Reyna and Brainerd, 2008 ; Hoffrage et al, 2015 ; Johnson and Tubau, 2015 , 2017 ; Sirota et al, 2015 ; Talboy and Schneider, 2017 , 2018a , b ).…”

“…The goal of the present research is to extend our previous efforts to identify factors that prevent reasoners from being ready to recognize and apply Bayes theorem to update probabilities based on diagnostic test information (e.g., Talboy and Schneider, 2018a , b ). To do this, we deconstruct the problem into a simpler format that allows an assessment of whether and when reasoners can recognize and apply the needed information to infer the PPV.…”

“…Factors associated with the low accuracy rates observed in Bayesian reasoning tasks are broadly categorized as either representational or computational difficulties ( Johnson and Tubau, 2015 , 2017 ; Talboy and Schneider, 2018a , b ). Difficulties with how the problem is cognitively represented by reasoners are typically attributed to how the components of the problem relate to each other, which is more or less apparent depending on the formulation of the problem.…”

“…Across a variety of problem types, research suggests that inexperienced reasoners will often rely on the initial problem structure and organization to guide their approach to solution ( Chi et al, 1981b ; Talboy and Schneider, 2018a ). Representational and computational difficulties are compounded in the standard Bayesian problem representation wherein the problem starts from one reference point (the condition) but the question asks reasoners to assess the information from another reference point (the test; Johnson and Tubau, 2013 , 2017 ; Pighin et al, 2018 ; Talboy and Schneider, 2018a , b ).…”

“…We propose that this reference dependence plays a major role in the solutions that reasoners generate. We also posit that both reference dependence and difficulties discerning the structure of the problem contribute to what we refer to as a value selection bias , wherein problem solvers who are uncertain about the correct response opt for one of the values present in the problem rather than performing computations when they cannot readily discern how to reach an accurate solution ( Gigerenzer et al, 1991 ; Gigerenzer and Hoffrage, 1995 ; Cosmides and Tooby, 1996 ; Galesic et al, 2009 ; Wolfe et al, 2013 ; Talboy and Schneider, 2017 , 2018a ). Similar biases have been observed, for instance, in conditional reasoning problems such as the Wason selection task ( Wason, 1966 ; Evans and Lynch, 1973 ; Evans, 1998 ; Evans et al, 2003 ).…”