Background Many healthy women consider genetic testing for breast cancer risk, yet BRCA testing issues are complex. Objective Determining whether an intelligent tutor, BRCA Gist, grounded in fuzzy-trace theory (FTT), increases gist comprehension and knowledge about genetic testing for breast cancer risk, improving decision-making. Design In two experiments, 410 healthy undergraduate women were randomly assigned to one of three groups: an online module using a web-based tutoring system (BRCA Gist) that uses artificial intelligence technology, a second group read highly similar content from the NCI web site, and a third completed an unrelated tutorial. Intervention BRCA Gist applied fuzzy trace theory and was designed to help participants develop gist comprehension of topics relevant to decisions about BRCA genetic testing, including how breast cancer spreads, inherited genetic mutations, and base rates. Measures We measured content knowledge, gist comprehension of decision-relevant information, interest in testing, and genetic risk and testing judgments. Results Control knowledge scores ranged from 54% to 56%, NCI improved significantly to 65% and 70%, and BRCA Gist improved significantly more to 75% and 77%, p<.0001. BRCA Gist scored higher on gist comprehension than NCI and control, p<.0001. Control genetic risk-assessment mean was 48% correct; BRCA Gist (61%), and NCI (56%) were significantly higher, p<.0001. BRCA Gist participants recommended less testing for women without risk factors (not good candidates), (24% and 19%) than controls (50%, both experiments) and NCI, (32%) Experiment 2, p<.0001. BRCA Gist testing interest was lower than controls, p<.0001. Limitations BRCA Gist has not been tested with older women from diverse groups. Conclusions Intelligent tutors, such as BRCA Gist, are scalable, cost effective ways of helping people understand complex issues, improving decision-making.

The goal of Intelligent Tutoring Systems (ITS) that interact in natural language is to emulate the benefits a well-trained human tutor provides to students, by interpreting student answers and appropriately responding to encourage elaboration. BRCA Gist is an ITS developed using AutoTutor Lite, a web-based version of AutoTutor. Fuzzy-Trace Theory theoretically motivated the development of BRCA Gist, which engages people in tutorial dialogues to teach them about genetic breast cancer risk. We describe an empirical method to create tutorial dialogues and fine-tune the calibration of BRCA Gist’s semantic processing engine without a team of computer scientists. We created five interactive dialogues centered on pedagogic questions, such as “What should someone do if she receives a positive result for genetic risk of breast cancer?” This method involved an iterative refinement process of repeated testing with different texts, and successively making adjustments to the tutor’s expectations and settings to improve performance. The goal of this method was to enable BRCA Gist to interpret and respond to answers in a manner that best facilitates learning. We developed a method to analyze the efficacy of the tutor’s dialogues. We found that BRCA Gist’s assessment of participants’ answers was highly correlated with the quality of answers found by trained human judges using a reliable rubric. Dialogue quality between users and BRCA Gist, predicted performance on a breast cancer risk knowledge test completed after the tutor. The appropriateness of BRCA Gist feedback also predicted the quality of answers and breast cancer risk knowledge test scores.

The intelligent tutoring system (ITS) BRCA Gist is a Web-based tutor developed using the Shareable Knowledge Objects (SKO) platform that uses latent semantic analysis to engage women in natural-language dialogues to teach about breast cancer risk. BRCA Gist appears to be the first ITS designed to assist patients' health decision making. Two studies provide fine-grained analyses of the verbal interactions between BRCA Gist and women responding to five questions pertaining to breast cancer and genetic risk. We examined how Bgist explanations^generated by participants during naturallanguage dialogues related to outcomes. Using reliable rubrics, scripts of the participants' verbal interactions with BRCA Gist were rated for content and for the appropriateness of the tutor's responses. Human researchers' scores for the content covered by the participants were strongly correlated with the coverage scores generated by BRCA Gist, indicating that BRCA Gist accurately assesses the extent to which people respond appropriately. In Study 1, participants' performance during the dialogues was consistently associated with learning outcomes about breast cancer risk. Study 2 was a field study with a more diverse population. Participants with an undergraduate degree or less education who were randomly assigned to BRCA Gist scored higher on tests of knowledge than those assigned to the National Cancer Institute website or than a control group. We replicated findings that the more expected content that participants included in their gist explanations, the better they performed on outcome measures. As fuzzy-trace theory suggests, encouraging people to develop and elaborate upon gist explanations appears to improve learning, comprehension, and decision making.

The BRCA Gist Intelligent Tutoring System helps women understand and make decisions about genetic testing for breast cancer risk. BRCA Gist is guided by Fuzzy-Trace Theory, (FTT) and built using AutoTutor Lite. It responds differently to participants depending on what they say. Seven tutorial dialogues requiring explanation and argumentation are guided by three FTT concepts: forming gist explanations in one’s own words, emphasizing decision-relevant information, and deliberating the consequences of decision alternatives. Participants were randomly assigned to BRCA Gist, a control, or impoverished BRCA Gist conditions removing gist explanation dialogues, argumentation dialogues, or FTT images. All BRCA Gist conditions performed significantly better than controls on knowledge, comprehension, and risk assessment. Significant differences in knowledge, comprehension, and fine-grained dialogue analyses demonstrate the efficacy of gist explanation dialogues. FTT images significantly increased knowledge. Providing more elements in arguments against testing correlated with increased knowledge and comprehension.