Improving Accuracy on Bayesian Inference Problems Using a Brief Tutorial

Abstract

Research suggests that most people struggle when asked to interpret the outcomes of diagnostic tests such as those presented as Bayesian inference problems. To help people interpret these difficult problems, we created a brief tutorial, requiring less than 10 minutes, that guided participants through the creation of an aid (either graph or table) based on an example inference problem and then showed the correct way to calculate the positive predictive value of the problem (i.e., likelihood that positive tests correctly indicate presence of condition). Approximately 70% of those in each training condition found the correct response on at least one problem in the format for which they were trained. Just under 55% of those in the control condition (i.e., no training) were able to find the correct response on at least one table or graph problem. We demonstrated a relationship between numeracy and performance on both problem formats, although we did not find evidence for a relationship between graph literacy and performance for either problem format. Potential improvements to and applications of the tutorial are discussed.

How to Improve Performance in Bayesian Inference Tasks: A Comparison of Five Visualizations

“…(2011), whereas area-proportionality did not prove to be a facilitating factor in Micallef et al. (2012) or Talboy and Schneider (2017).…”

“…One idea about beneficial graphical properties of visualization refers to representing the statistical information area proportionally (e.g., Tsai et al., 2011; Micallef et al., 2012): “this accurate, proportional representation is considered a key feature of what makes a good visual aid” (Talboy and Schneider, 2017, p. 375). Theoretical arguments for area-proportional visualizations, such as the unit square (Figure 3B), are often formulated based on mathematical considerations: “Rectangular areas correspond to probabilities and can be used to calculate their numerical value and to determine the Bayes relation” (Oldford, 2003, p. 1).…”

Visualizing the Bayesian 2-test case: The effect of tree diagrams on medical decision making

“…There is another strategy for improving Bayesian reasoning in the 1-test case, namely, visualizing the statistical information. Some prominent visualizations that have been developed are Euler diagrams (e.g., [ 29 – 31 ]), roulette-wheel diagrams (e.g., [ 32 , 33 ]), frequency grids (e.g., [ 23 , 34 , 35 ]), Eikosograms (sometimes also called unit squares or mosaic plots ; e.g., [ 36 – 39 ]), icon arrays (e.g., [ 32 , 40 , 41 ]), 2×2-tables (e.g., [ 14 , 42 ]), and tree diagrams (e.g., [ 14 , 33 , 42 – 44 ]). For an overview of these visualizations, see [ 14 ], and for corresponding visualizations regarding the 2-test case, see Fig 1 .…”

Different Visualizations Cause Different Strategies When Dealing With Bayesian Situations

“…Further, our results can be used to improve trainings of Bayesian reasoning that are based on a double-tree diagram (Wassner, 2004) or a unit square (Talboy and Schneider, 2017). When using a double-tree diagram, a specific focus must be put on identifying the correct subset H ∩ D, and emphasizing the related node as representing the intersection set H ∩ D that allows for the set inclusion (H ∩ D) ⊆ D. When using a unit square, our results imply that a specific focus must be put on identification of the correct basic set, since most of the students found a correct strategy based on this identification.…”

“…A unit square (Eichler and Vogel, 2015; Figure 3B) representing a nested style (Khan et al, 2015) was also found to facilitate Bayesian reasoning (Oldford, 2003;Eichler, 2017, 2019;Talboy and Schneider, 2017). In a unit square, the set inclusion (H ∩ D) ⊆ D and (H ∩ D) ⊆ D as well as (H ∩ D) ⊆ H and (H ∩D) ⊆ H are presented in one row or in one column.…”