Background Many healthy women consider genetic testing for breast cancer risk, yet BRCA testing issues are complex. Objective Determining whether an intelligent tutor, BRCA Gist, grounded in fuzzy-trace theory (FTT), increases gist comprehension and knowledge about genetic testing for breast cancer risk, improving decision-making. Design In two experiments, 410 healthy undergraduate women were randomly assigned to one of three groups: an online module using a web-based tutoring system (BRCA Gist) that uses artificial intelligence technology, a second group read highly similar content from the NCI web site, and a third completed an unrelated tutorial. Intervention BRCA Gist applied fuzzy trace theory and was designed to help participants develop gist comprehension of topics relevant to decisions about BRCA genetic testing, including how breast cancer spreads, inherited genetic mutations, and base rates. Measures We measured content knowledge, gist comprehension of decision-relevant information, interest in testing, and genetic risk and testing judgments. Results Control knowledge scores ranged from 54% to 56%, NCI improved significantly to 65% and 70%, and BRCA Gist improved significantly more to 75% and 77%, p<.0001. BRCA Gist scored higher on gist comprehension than NCI and control, p<.0001. Control genetic risk-assessment mean was 48% correct; BRCA Gist (61%), and NCI (56%) were significantly higher, p<.0001. BRCA Gist participants recommended less testing for women without risk factors (not good candidates), (24% and 19%) than controls (50%, both experiments) and NCI, (32%) Experiment 2, p<.0001. BRCA Gist testing interest was lower than controls, p<.0001. Limitations BRCA Gist has not been tested with older women from diverse groups. Conclusions Intelligent tutors, such as BRCA Gist, are scalable, cost effective ways of helping people understand complex issues, improving decision-making.

The goal of Intelligent Tutoring Systems (ITS) that interact in natural language is to emulate the benefits a well-trained human tutor provides to students, by interpreting student answers and appropriately responding to encourage elaboration. BRCA Gist is an ITS developed using AutoTutor Lite, a web-based version of AutoTutor. Fuzzy-Trace Theory theoretically motivated the development of BRCA Gist, which engages people in tutorial dialogues to teach them about genetic breast cancer risk. We describe an empirical method to create tutorial dialogues and fine-tune the calibration of BRCA Gist’s semantic processing engine without a team of computer scientists. We created five interactive dialogues centered on pedagogic questions, such as “What should someone do if she receives a positive result for genetic risk of breast cancer?” This method involved an iterative refinement process of repeated testing with different texts, and successively making adjustments to the tutor’s expectations and settings to improve performance. The goal of this method was to enable BRCA Gist to interpret and respond to answers in a manner that best facilitates learning. We developed a method to analyze the efficacy of the tutor’s dialogues. We found that BRCA Gist’s assessment of participants’ answers was highly correlated with the quality of answers found by trained human judges using a reliable rubric. Dialogue quality between users and BRCA Gist, predicted performance on a breast cancer risk knowledge test completed after the tutor. The appropriateness of BRCA Gist feedback also predicted the quality of answers and breast cancer risk knowledge test scores.

Two studies examined semantic coherence and internal inconsistency fallacies in conditional probability estimation. Problems reflected five distinct relationships between two sets: identical sets, mutually exclusive sets, subsets, overlapping sets, and independent sets (a special case of overlapping sets). Participants estimated P(A), P(B), P(A|B), and P(B|A). Inconsistency occurs when this constellation of estimates does not conform to Bayes' theorem. Semantic coherence occurs when this constellation of estimates is consistent with the depicted relationship among sets. Fuzzy-trace theory predicts that people have difficulty with overlapping sets and subsets because they require class-inclusion reasoning. On these problems, people are vulnerable to denominator neglect, the tendency to ignore relevant denominators, making the gist more difficult to discern. Independent sets are simplified by the gist understanding that P(A) provides no information about P(B), and thus, P(A|B) = P(A). The gist for identical sets is that P(A|B) = 1.0, and the gist of mutually exclusive sets is that P(A|B) = 0. In Study 1, identical, mutually exclusive, and independent sets yielded superior performance (in internal inconsistency and semantic coherence) than subsets and overlapping sets. For subsets and overlapping sets, interventions clarifying appropriate denominators generally improved semantic coherence and inconsistency, including teaching people to use Euler diagrams, 2 Â 2 tables, or relative frequencies. In Study 2, with problems about breast cancer and BRCA mutations, there was a strong correlation between inconsistency in conditional probability estimation and conjunction fallacies of joint probability estimation, suggesting that similar fallacious reasoning processes produce these errors.

Two experiments tested predictions derived from the Probability Theory + Variation (PTV) model. PTV model assumes that judgments follow probability theory, but systematic errors arise from noise in the judgments. Experiment 1 compared the PTV model to a configural weighted averaging model in joint probability judgment and found more support for the PTV model in diagnostic cases. Specifically, noise was negatively correlated with semantic coherence and conjunction and disjunction fallacies increased when order effects produced more noise in conjunctions and disjunctions. Consistent with both models, judgments adhered stochastically to the addition law. Contrary to the integration rules of the PTV model, we failed to find increased noise in disjunctions compared to conjunctions. Experiment 2 tested predictions of the PTV model for conditional probability judgment. Consistent with the PTV model, noise was negatively correlated with semantic coherence in conditional probabilities and judgments adhered stochastically to Bayes' theorem. Conversion errors were generally more prevalent than conditional reversals, a finding that is not fully consistent with the PTV model. In general, the quantitative fit of the PTV model was relatively better for overlapping and subset problems compared to identical, independent and mutually exclusive problems, especially for semantic coherence. Copyright © 2014 John Wiley & Sons, Ltd.