Machine Explanations and Human Understanding

Chen, Chacha; Shi, Feng; Sharma, Vishal; Tan, Chenhao

doi:10.48550/arxiv.2202.04092

Cited by 5 publications

(2 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This provides a possible path for better signaling groundtruth labels to help decision-makers avoid over-reliance and make better decisions. This path resonates with a recent theoretical work by Chen et al [19], which suggests that feature-based explanations can only reveal model decision boundaries (how the model makes decisions), and it is by their contrast with human intuitions about the task boundaries (which features should contribute to the outcome) can one detect model errors. We may in fact view the gray-out words as such contrasts.…”

Section: Discussionsupporting

confidence: 84%

Selective Explanations: Leveraging Human Input to Align Explainable AI

Lai¹,

Zhang²,

Chen³

et al. 2023

Preprint

View full text Add to dashboard Cite

While a vast collection of explainable AI (XAI) algorithms have been developed in recent years, they are often criticized for significant gaps with how humans produce and consume explanations. As a result, current XAI techniques are often found to be hard to use and lack effectiveness. In this work, we attempt to close these gaps by making AI explanations selective-a fundamental property of human explanations-by selectively presenting a subset from a large set of model reasons based on what aligns with the recipient's preferences. We propose a general framework for generating selective explanations by leveraging human input on a small sample.This framework opens up a rich design space that accounts for different selectivity goals, types of input, and more. As a showcase, we use a decision-support task to explore selective explanations based on what the decision-maker would consider relevant to the decision task. We conducted two experimental studies to examine three out of a broader possible set of paradigms based on our proposed framework: in Study 1, we ask the participants to provide their own input to generate selective explanations, with either open-ended or critique-based input. In Study 2, we show participants selective explanations based on input from a panel of similar users (annotators). Our experiments demonstrate the promise of selective explanations in reducing over-reliance on AI and improving decision outcomes and subjective perceptions of the AI, but also paint a nuanced picture that attributes some of these positive effects to the opportunity to provide one's own input to augment AI explanations. Overall, our work proposes a novel XAI framework inspired by human communication behaviors and demonstrates its potentials to encourage future work to better align AI explanations with human production and consumption of explanations. CCS Concepts: • Human-centered computing → Collaborative and social computing; • Computing methodologies → Artificial intelligence; • Applied computing → Law, social and behavioral sciences.

show abstract

Section: Discussionsupporting

confidence: 84%

Selective Explanations: Leveraging Human Input to Align Explainable AI

Lai¹,

Zhang²,

Chen³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Indeed, it is essential to provide accurate and understandable explanations as poor explanations can sometimes be even worse than no explanation at all [80] and may also generate undesired bias in the users [81,82]. As a consequence, properly structuring [83] and evaluating the interpretability and effectiveness of explanations requires a deep understanding of the ways in which humans interpret and understand them, while also accounting for the relationship between human understanding and model explanations [84,85]. For such reasons, the explainable AI research field spreads from IT-related fields, such as computer science and machine learning, to a variety of humancentred disciplines, such as psychology, philosophy, and decision making [86].…”

Section: Understanding the Human's Perspective In Explainable Aimentioning

confidence: 99%

The Role of Human Knowledge in Explainable AI

Tocchetti

Brambilla

2022

Data

View full text Add to dashboard Cite

As the performance and complexity of machine learning models have grown significantly over the last years, there has been an increasing need to develop methodologies to describe their behaviour. Such a need has mainly arisen due to the widespread use of black-box models, i.e., high-performing models whose internal logic is challenging to describe and understand. Therefore, the machine learning and AI field is facing a new challenge: making models more explainable through appropriate techniques. The final goal of an explainability method is to faithfully describe the behaviour of a (black-box) model to users who can get a better understanding of its logic, thus increasing the trust and acceptance of the system. Unfortunately, state-of-the-art explainability approaches may not be enough to guarantee the full understandability of explanations from a human perspective. For this reason, human-in-the-loop methods have been widely employed to enhance and/or evaluate explanations of machine learning models. These approaches focus on collecting human knowledge that AI systems can then employ or involving humans to achieve their objectives (e.g., evaluating or improving the system). This article aims to present a literature overview on collecting and employing human knowledge to improve and evaluate the understandability of machine learning models through human-in-the-loop approaches. Furthermore, a discussion on the challenges, state-of-the-art, and future trends in explainability is also provided.

show abstract