How level of explanation detail affects human performance in interpretable intelligent systems: A study on explainable fact checking

Abstract

Explainable artificial intelligence (XAI) systems aim to provide users with information to help them better understand computational models and reason about why outputs were generated. However, there are many different ways an XAI interface might present explanations, which makes designing an appropriate and effective interface an important and challenging task. Our work investigates how different types and amounts of explanatory information affect user ability to utilize explanations to understand system behavior and improve task performance. The presented research employs a system for detecting the truthfulness of news statements. In a controlled experiment, participants were tasked with using the system to assess news statements as well as to learn to predict the output of the AI. Our experiment compares various levels of explanatory information to contribute empirical data about how explanation detail can influence utility. The results show that more explanation information improves participant understanding of AI models, but the benefits come at the cost of time and attention needed to make sense of the explanation.

Guided By AI: Navigating Trust, Bias, and Data Exploration in AI‐Guided Visual Analytics

“…Although the space of explanation techniques is vast, we opted for variations of simple prediction‐specific transparency methods for users to understand – the most influential keyword and confidence value. Inspired by prior studies [ZLB20, LMY*21, WY21], we posit that providing the most influential keyword and confidence value may improve trust calibration by giving users an indication to increase their situational awareness of the ai 's performance. We define an influential keyword to be one which if eliminated, decreases the probability of a post being relevant by the largest amount.…”

“…Similarly, Dietvorst et al [DSM18] communicated a model's uncertainty through the outright disclosure of the model's average error rate by a text description (i.e., “the model has an average error rate of x”). Linder et al [LMY*21] explored how the type and amount of explanations affect users' understanding and performance on a fact‐checking task. More detailed explanations such as providing examples of alternative statements with the same classification and information about the influence of the statement's metadata led the users to a better understanding of the ai suggestions.…”

“…However, these techniques may not necessarily increase human‐ ai performance. For example, too much information about the ai could lead to worse performance or decision‐making outcomes [LMY*21].…”

What Does Evaluation of Explainable Artificial Intelligence Actually Tell Us? A Case for Compositional and Contextual Validation of XAI Building Blocks

“…While many studies assess the effectiveness of an explanation type, content and presentation medium, they often overlook the inherent correctness and trustworthiness of the underlying explanatory insight, or rather the pervasive lack thereof [43,65,80]. This inspires an alternative, diagnostic conceptualisation of XAI, which focuses on providing users with rigorously tested and well-specified insights into a predictive model instead of attempting to solve the ill-defined "black box" problem [14].…”

The Role of Human Knowledge in Explainable AI

“…In conclusion, achieving a high level of transparency is not always beneficial to improving the user's understanding [5,81]. Indeed, providing complex or a large number of explanations would generate a trade-off between their understandability and the time required by human interpreters to interpret them [42,90]. Consequently, it is necessary to comprehend the proper level of transparency, explanation complexity and quantity, even in simple cases [91].…”