Investigating the Impact of Explanation on Repairing Trust in Ai Diagnostic Systems for Re-Diagnosis

Alam, Lamia

doi:10.37099/mtu.dc.etdr/1029

Cited by 4 publications

(8 citation statements)

References 100 publications

(108 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One such type of explanation is whether the goal of the explanation is to inform about the diagnostic process, versus justify why a particular diagnosis was made. These explanation types are respectively referred to as "global" and "local" explanations [46][47][48][49].In general, Alam [44] found that physicians report using both methods; sometimes they explain how a particular disease or diagnostic process works; other times they justify why a particular diagnosis is given based on evidence (symptoms, test results, history, etc. ).…”

Section: Methods For Providing Explanationsmentioning

confidence: 99%

“…Another important distinction is the means by which an explanation is provided. Alam [44] also found that physicians' explanations mapped onto many of the explanation types studied in the XAI literature, including casebased information and examples [50,51], analogies [52,53], logical arguments [54,55], visualizing imagery and highlighting important aspects [56,57]. For imagery, AI healthcare systems may use graphs to show the relative probability of different outcomes or the relative importance of different symptoms for those outcomes, which is more akin to how the LIME algorithm [58] works for diagnostic features.…”

Section: Methods For Providing Explanationsmentioning

confidence: 99%

“…One final observation we make is that it is a mistake to think about AI diagnosis as merely a classification problem that determines a disease or condition based on symptoms and signs. For example, Alam [44] identified how diagnosis involves explaining why and how the AI is making the diagnosis. As in our scenario, an error is not necessarily an actual mistake, it might be the most likely outcome that happens to be wrong for an individual.…”

Section: Lesson: Diagnosis Is Not Simply Classificationmentioning

confidence: 99%

“…One approach we have pursued to study explainable AI (XAI) in healthcare is to understand the types of explanations real physicians offer when they interact with patients. For example, Alam [44] conducted an interview study with physicians to document how they explained diagnoses to the patients. The results suggest that physicians use a variety of explanation methods, which are dependent on context, including time (i.e., early or later in diagnosis) and the patient or patient's advocate's identity (including cultural, education, age, and other concerns).…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Examining the effect of explanation on satisfaction and trust in AI diagnostic systems

Alam

2021

BMC Med Inform Decis Mak

Self Cite

View full text Add to dashboard Cite

Background Artificial Intelligence has the potential to revolutionize healthcare, and it is increasingly being deployed to support and assist medical diagnosis. One potential application of AI is as the first point of contact for patients, replacing initial diagnoses prior to sending a patient to a specialist, allowing health care professionals to focus on more challenging and critical aspects of treatment. But for AI systems to succeed in this role, it will not be enough for them to merely provide accurate diagnoses and predictions. In addition, it will need to provide explanations (both to physicians and patients) about why the diagnoses are made. Without this, accurate and correct diagnoses and treatments might otherwise be ignored or rejected. Method It is important to evaluate the effectiveness of these explanations and understand the relative effectiveness of different kinds of explanations. In this paper, we examine this problem across two simulation experiments. For the first experiment, we tested a re-diagnosis scenario to understand the effect of local and global explanations. In a second simulation experiment, we implemented different forms of explanation in a similar diagnosis scenario. Results Results show that explanation helps improve satisfaction measures during the critical re-diagnosis period but had little effect before re-diagnosis (when initial treatment was taking place) or after (when an alternate diagnosis resolved the case successfully). Furthermore, initial “global” explanations about the process had no impact on immediate satisfaction but improved later judgments of understanding about the AI. Results of the second experiment show that visual and example-based explanations integrated with rationales had a significantly better impact on patient satisfaction and trust than no explanations, or with text-based rationales alone. As in Experiment 1, these explanations had their effect primarily on immediate measures of satisfaction during the re-diagnosis crisis, with little advantage prior to re-diagnosis or once the diagnosis was successfully resolved. Conclusion These two studies help us to draw several conclusions about how patient-facing explanatory diagnostic systems may succeed or fail. Based on these studies and the review of the literature, we will provide some design recommendations for the explanations offered for AI systems in the healthcare domain.

show abstract

Section: Methods For Providing Explanationsmentioning

confidence: 99%

Section: Methods For Providing Explanationsmentioning

confidence: 99%

Section: Lesson: Diagnosis Is Not Simply Classificationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Examining the effect of explanation on satisfaction and trust in AI diagnostic systems

Alam

2021

BMC Med Inform Decis Mak

Self Cite

View full text Add to dashboard Cite

show abstract

“…Another such property is scope or focus (see Doshi-Velez & Kim, 2017;Wick & Slagle, 1989), describing whether an explanation refers to specific cases (local) or large-scale patterns and operations of the system (global). Alam (2020) showed how this scope can impact different aspects of satisfaction, and so it is important for heuristic evaluation. Related to this is explanation form, determined by the kind of question the explanation answers.…”

Section: The Goodness Criteria For Heuristic Evaluationmentioning

confidence: 99%

Assessing Collaborative Explanations of AI using Explanation Goodness Criteria

Mamun

Baker

Malinowski

et al. 2021

Proceedings of the Human Factors and Ergonomics Society Annual

View full text Add to dashboard Cite

Explainable AI represents an increasingly important category of systems that attempt to support human understanding and trust in machine intelligence and automation. Typical systems rely on algorithms to help understand underlying information about decisions and establish justified trust and reliance. Researchers have proposed using goodness criteria to measure the quality of explanations as a formative evaluation of an XAI system, but these criteria have not been systematically investigated in the literature. To explore this, we present a novel collaborative explanation system (CXAI) and propose several goodness criteria to evaluate the quality of its explanations. Results suggest that the explanations provided by this system are typically correct, informative, written in understandable ways, and focus on explanation of larger scale data patterns than are typically generated by algorithmic XAI systems. Implications for how these criteria may be applied to other XAI systems are discussed.

show abstract

Examining Physicians’ Explanatory Reasoning in Re-Diagnosis Scenarios for Improving AI Diagnostic Systems

Alam

2022

Journal of Cognitive Engineering and Decision Making

Self Cite

View full text Add to dashboard Cite

AI systems are increasingly being developed to provide the first point of contact for patients. These systems are typically focused on question-answering and integrating chat systems with diagnostic algorithms, but are likely to suffer from many of the same deficiencies in explanation that have plagued medical diagnostic systems since the 1970s ( Shortliffe, 1979 ). To provide better guidance about how such systems should approach explanations, we report on an interview study in which we identified explanations that physicians used in the context of re-diagnosis or a change in diagnosis. Seven current and former physicians with a variety of specialties and experience were recruited to take part in the interviews. Several high-level observations were made by reviewing the interview notes. Nine broad categories of explanation emerged from the thematic analysis of the explanation contents. We also present these in a diagnosis meta-timeline that encapsulates many of the commonalities we saw across diagnoses during the interviews. Based on the results, we provided some design recommendations to consider for developing diagnostic AI systems. Altogether, this study suggests explanation strategies, approaches, and methods that might be used by medical diagnostic AI systems to improve user trust and satisfaction with these systems.

show abstract

Investigating the Impact of Explanation on Repairing Trust in Ai Diagnostic Systems for Re-Diagnosis

Cited by 4 publications

References 100 publications

Examining the effect of explanation on satisfaction and trust in AI diagnostic systems

Examining the effect of explanation on satisfaction and trust in AI diagnostic systems

Assessing Collaborative Explanations of AI using Explanation Goodness Criteria

Examining Physicians’ Explanatory Reasoning in Re-Diagnosis Scenarios for Improving AI Diagnostic Systems

Contact Info

Product

Resources

About