An explanation space to align user studies with the technical development of Explainable AI

Cabour, Garrick; Morales-Forero, Andrés; Ledoux, Élise; Bassetto, Samuel

doi:10.1007/s00146-022-01536-6

Cited by 8 publications

(6 citation statements)

References 63 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As some researchers note, an explanation can be correct but at the same time it may not be any good or useful (for a given task) [1], or simply put "one explanation does not fit all" [69]. To be effective, explanations should align with the explainees' objectives and provide them with the required information, thus be helpful in a given situation as outside of it the explanatory insights may simply become irrelevant or meaningless [12,52]. This crucial role of context, nonetheless, seems to be rarely acknowledged when choosing metrics and protocols for evaluating XAI techniques, which appears to be one of the main reasons for the inconsistent findings highlighted in the previous paragraph.…”

Section: Evaluation Deficienciesmentioning

confidence: 99%

“…Triggered by expectation failures (e.g., a surprising event), learning needs or task requirements, it allows to adjust misconceptions, reconcile differences and resolve disagreements by helping a person to develop correct knowledge, beliefs and understanding of the explained phenomenon [20]. Such a perspective entails a separation between • the explanatory artefacts, i.e., the type, content, presentation format and provision mechanism of an explanation determined by what needs to be explained, and • the act of explaining, i.e., the explanatory process governed by how to explain it, which draws an implicit distinction between user interfaces and explainability [1,12,20,31,33,39].…”

Section: Evaluation Deficienciesmentioning

confidence: 99%

See 1 more Smart Citation

What Does Evaluation of Explainable Artificial Intelligence Actually Tell Us? A Case for Compositional and Contextual Validation of XAI Building Blocks

Sokol,

Vogt

2024

Extended Abstracts of the CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

Despite significant progress, evaluation of explainable artificial intelligence remains elusive and challenging. In this paper we propose a fine-grained validation framework that is not overly reliant on any one facet of these sociotechnical systems, and that recognises their inherent modular structure: technical building blocks, userfacing explanatory artefacts and social communication protocols. While we concur that user studies are invaluable in assessing the quality and effectiveness of explanation presentation and delivery strategies from the explainees' perspective in a particular deployment context, the underlying explanation generation mechanisms require a separate, predominantly algorithmic validation strategy that accounts for the technical and human-centred desiderata of their (numerical) outputs. Such a comprehensive sociotechnical utility-based evaluation framework could allow to systematically reason about the properties and downstream influence of different building blocks from which explainable artificial intelligence systems are composed -accounting for a diverse range of their engineering and social aspects -in view of the anticipated use case.

show abstract

Section: Evaluation Deficienciesmentioning

confidence: 99%

Section: Evaluation Deficienciesmentioning

confidence: 99%

What Does Evaluation of Explainable Artificial Intelligence Actually Tell Us? A Case for Compositional and Contextual Validation of XAI Building Blocks

Sokol,

Vogt

2024

Extended Abstracts of the CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

show abstract

“…Another area of inquiry is measuring the adequacy of explanations [37]. The insights from "explanation science" can be used to build better explanations [35,38,46]. Since explanation is a social transaction between the explainer and the end user, additional research [35,46] is aimed at tailoring explanations to the cognitive landscape of the end user (i.e.…”

Section: Progress In Xaimentioning

confidence: 99%

“…These methods have been applied to improve explainability in different contexts [13,20,[41][42][43][44][45]. We call these explanation methods explanation agents (see [46]). Das and Rad [33] have reviewed different approaches to creating explanation agents, including perturbation-based methods (SHAP, LIME) and gradient-based methods (saliency maps).…”

Section: Progress In Xaimentioning

confidence: 99%

“…Cabour et al [46] have suggested that explanations should reside in a formal explanation space with specified elements, such as classifier agents and explainer agents, participants such as the explainer and the end-user, and special consideration of the cognitive capabilities of the end-user. Like Cabour et al [46], we use the term formal explanation space to refer to an assembly of explanation agents that improves the interpretability of machine learning algorithms.…”

Section: Progress In Xaimentioning

confidence: 99%

See 1 more Smart Citation

A Formal Explanation Space for the Simultaneous Clustering of Neurology Phenotypes

Yelugam,

Hier,

Obafemi-Ajayi

et al. 2023

Preprint

View full text Add to dashboard Cite

Objective: Clustering is applied to biomedical datasets to identify meaningful subgroups of patients, proteins, genes, and diseases. Explainable AI (XAI) brings transparency and interpretability to the formation, composition, and quality of these clusters. This study creates a formal explanation space to enhance the interpretability of clusters of neurology phenotypes. Methods: Subjects with dementia, movement disorders, and multiple sclerosis were clustered by neurological phenotype using spectral methods. To improve the interpretability of the clusters, we created an explanation space that described the data, explained the algorithm, evaluated cluster separation and quality, identified influential features, visualized cluster composition, and assessed biological plausibility. Results: Text and equations were used to explain clustering algorithms. Cluster quality was evaluated with validity indices. The t-SNE plots illustrate cluster separation. Influential features were identified from SHAP plots. The cluster composition was visualized with heat maps and word clouds. Expert opinion assessed biological relevance. Spectral coclustering yielded clusters with higher validity indices and biological plausibility than spectral biclustering. Conclusions: When biomedical data undergo simultaneous clustering, a formal explanation space can improve the transparency of the methods and interpretability of the results.

show abstract

XAI is in trouble

Weber,

Johs,

Goel

et al. 2024

AI Magazine

View full text Add to dashboard Cite

Researchers focusing on how artificial intelligence (AI) methods explain their decisions often discuss controversies and limitations. Some even assert that most publications offer little to no valuable contributions. In this article, we substantiate the claim that explainable AI (XAI) is in trouble by describing and illustrating four problems: the disagreements on the scope of XAI, the lack of definitional cohesion, precision, and adoption, the issues with motivations for XAI research, and limited and inconsistent evaluations. As we delve into their potential underlying sources, our analysis finds these problems seem to originate from AI researchers succumbing to the pitfalls of interdisciplinarity or from insufficient scientific rigor. Analyzing these potential factors, we discuss the literature at times coming across unexplored research questions. Hoping to alleviate existing problems, we make recommendations on precautions against the challenges of interdisciplinarity and propose directions in support of scientific rigor.

show abstract

An explanation space to align user studies with the technical development of Explainable AI

Cited by 8 publications

References 63 publications

What Does Evaluation of Explainable Artificial Intelligence Actually Tell Us? A Case for Compositional and Contextual Validation of XAI Building Blocks

What Does Evaluation of Explainable Artificial Intelligence Actually Tell Us? A Case for Compositional and Contextual Validation of XAI Building Blocks

A Formal Explanation Space for the Simultaneous Clustering of Neurology Phenotypes

XAI is in trouble

Contact Info

Product

Resources

About