Retrospective Frequency Formats Promote Consistent Experience‐Based Bayesian Judgments

Abstract

On the basis of their experiences with pregnant patients in their practice, obstetrician/gynecologists estimated the posterior probability of Down syndrome given a positive screening result. They also estimated the base rate of Down syndrome in their practice, along with the hit and false alarm rates for the screening test; for each subject, these numbers were combined to calculate a posterior probability to which the initial estimated posterior probability could be compared. Physicians gave highly consistent estimates when asked to think about their past experiences in terms of event frequencies. However, those told to respond using single event probabilities or to use past experiences to predict prospective frequencies gave inconsistent Bayesian estimates. Thus, when making Bayesian judgments based on real life experience, natural frequency formats only lead to better judgments, compared with single event probability formats, if people think retrospectively, not when using past experiences to make prospective predictions.

Why Can Only 24% Solve Bayesian Reasoning Problems in Natural Frequencies: Frequency Phobia in Spite of Probability Blindness

“…We speculate that such an Einstellung might be enhanced by the still widespread idea that natural frequencies are not “mathematically correct” enough to actually work with in high school and university contexts. As a consequence, participants who might actually notice a possible solution of the Bayesian reasoning task based on a frequency algorithm might still rely on probabilities due to a certain kind of “phobia” to use natural frequencies for their calculations (for a discussion on the impact of affect on overcoming fixed mindsets, see Haager et al, 2014 )—despite the ever-growing body of research pointing to the beneficial effects of the frequency concept (e.g., Gigerenzer and Hoffrage, 1995 ; Barbey and Sloman, 2007 ; Micallef et al, 2012 ; Obrecht et al, 2012 ; Ottley et al, 2016 ; McDowell and Jacobs, 2017 ).…”

“…Cognitive biases and judgment errors associated with different numerical information are, for example, size effect and distance effect (Moyer and Landauer, 1967 ). Finally, details of the representation and framing of the problem text can affect performance in Bayesian reasoning situations (Obrecht et al, 2012 ). Ottley et al ( 2016 ), for example, were able to show that specific problem formulations (e.g., providing all numerical information in context of the task, that is, not only base rate, sensitivity, and false alarm rate but also the probability or frequency of their respective complement) influence accuracies significantly.…”

Sample representativeness affects whether judgments are influenced by base rate or sample size

“…Bayesian reasoning is improved, however, when data are presented as unstandardized natural frequencies that express probability in terms of subsets within a greater super set (Brase, 2008;Gigerenzer & Hoffrage, 1995;Obrecht, Anderson, Schulkin, & Chapman, 2012). For example, analogous to the problem above:…”

Aiding Risk Information learning through Simulated Experience (ARISE): A Comparison of the Communication of Screening Test Information in Explicit and Simulated Experience Formats

“…For example, a clinician may gradually learn the prevalence of disease in a patient population through experiencing individual patient cases. 18 The ecological hypothesis posits that humans have evolved to accumulate naturally occurring frequency information through experience to learn about the environment. 19,20 It is argued that natural frequencies foster insight into Bayesian reasoning (i.e., estimating conditional probabilities) because humans have adapted to process these representations.…”

“…22 In particular, descriptive formats are associated with overestimating the PPV, while experience formats are associated with more accurate PPV estimates and less systematic errors. 4,5,12,18 In addition to accuracy, experience formats have been shown to change subjective risk perception and intentions to screen. Specifically, learning from experience decreases worry about Down syndrome following a positive test result as well as interest in screening.…”