Semantic Coherence and Inconsistency in Estimating Conditional Probabilities

Abstract

Two studies examined semantic coherence and internal inconsistency fallacies in conditional probability estimation. Problems reflected five distinct relationships between two sets: identical sets, mutually exclusive sets, subsets, overlapping sets, and independent sets (a special case of overlapping sets). Participants estimated P(A), P(B), P(A|B), and P(B|A). Inconsistency occurs when this constellation of estimates does not conform to Bayes' theorem. Semantic coherence occurs when this constellation of estimates is consistent with the depicted relationship among sets. Fuzzy-trace theory predicts that people have difficulty with overlapping sets and subsets because they require class-inclusion reasoning. On these problems, people are vulnerable to denominator neglect, the tendency to ignore relevant denominators, making the gist more difficult to discern. Independent sets are simplified by the gist understanding that P(A) provides no information about P(B), and thus, P(A|B) = P(A). The gist for identical sets is that P(A|B) = 1.0, and the gist of mutually exclusive sets is that P(A|B) = 0. In Study 1, identical, mutually exclusive, and independent sets yielded superior performance (in internal inconsistency and semantic coherence) than subsets and overlapping sets. For subsets and overlapping sets, interventions clarifying appropriate denominators generally improved semantic coherence and inconsistency, including teaching people to use Euler diagrams, 2 Â 2 tables, or relative frequencies. In Study 2, with problems about breast cancer and BRCA mutations, there was a strong correlation between inconsistency in conditional probability estimation and conjunction fallacies of joint probability estimation, suggesting that similar fallacious reasoning processes produce these errors.

Efficacy of a Web-Based Intelligent Tutoring System for Communicating Genetic Risk of Breast Cancer

“…8 Thus, interventions applying FTT (incorporated in BRCA Gist) have been successful in helping people make sense of such relations, reducing biases and improving decision making. 8,30,55,61,62 …”

Individual Differences in Base Rate Neglect: A Fuzzy Processing Preference Index

“…For example, taking an individual differences approach, using the FPPI may shed light on Bayesian reasoning (Chater, Oaksford, Hahn, & Heit, 2010;Fisher & Wolfe, 2012) or even biases in reasoning and argumentation (Wolfe & Britt, 2008;Wolfe, Britt, & Butler, 2009). The FPPI can also be used in applied research, particularly in medical decision making (Peters, McCaul, Stefanek, & Nelson 2006;Reyna, 2008).…”

Communicating Numerical Risk

“…Results showed that in comparison to the control group, learning about Euler diagrams and 2 × 2 tables improved semantic coherence and decreased inconsistency, although understanding some set relations were helped more by particular representations. These findings demonstrated that clarifying appropriate denominators reduce errors by affecting class inclusion reasoning (Wolfe et al, in press). …”

“…Because ratios involve classes that overlap (e.g., number of people with disease or number of people with a positive test result), confusion arises from identifying the denominator (e.g., number of people with disease overall or number of people with positive test result overall). According to this theory, risk communication strategies should focus on making the denominator clear (discretely) by disentangling the classes that overlap, which reduces errors (Wolfe, Fisher, and Reyna, in press; Reyna & Mills, 2007). …”

“…To test multiple ways of disentangling classes and effectively presenting information about risk, Wolfe et al (in press) used 2 × 2 tables, Euler diagrams, or frequencies (using poker chips). In this study, 100 subjects were randomly assigned to learn about joint probabilities in one of the four conditions (the fourth being a control with no visual aid).…”