I Think I Get Your Point, AI! The Illusion of Explanatory Depth in Explainable AI

Chromik, Michael; Eiband, Malin; Buchner, Felicitas; Krüger, Adrian; Butz, Andreas

doi:10.1145/3397481.3450644

Cited by 76 publications

(53 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Model-agnostic local explanation methods, such as Shapley additive explanations (SHAP) ( Lundberg and Lee, 2017 ) and Local interpretable model-agnostic explanations (LIME) ( Ribeiro et al, 2016 ) have the potential to overcome this issue due to how the methods consistently and transparently quantify the input’s effect on prediction across most model types ( Lundberg et al, 2018 ). However, this leads back to the original criticism of describing the extraction of meaning post-hoc from the black box as a practice with potential bias [57], the ability to purposely engineer explanations ( Slack et al, 2020 ) and the likelihood of false conclusions being made by inexperienced users ( Chromik et al, 2021 ).…”

Section: Explainable (Interpretable) Machine Learning For Genotype To...mentioning

confidence: 99%

Plant Genotype to Phenotype Prediction Using Machine Learning

et al. 2022

View full text Add to dashboard Cite

Genomic prediction tools support crop breeding based on statistical methods, such as the genomic best linear unbiased prediction (GBLUP). However, these tools are not designed to capture non-linear relationships within multi-dimensional datasets, or deal with high dimension datasets such as imagery collected by unmanned aerial vehicles. Machine learning (ML) algorithms have the potential to surpass the prediction accuracy of current tools used for genotype to phenotype prediction, due to their capacity to autonomously extract data features and represent their relationships at multiple levels of abstraction. This review addresses the challenges of applying statistical and machine learning methods for predicting phenotypic traits based on genetic markers, environment data, and imagery for crop breeding. We present the advantages and disadvantages of explainable model structures, discuss the potential of machine learning models for genotype to phenotype prediction in crop breeding, and the challenges, including the scarcity of high-quality datasets, inconsistent metadata annotation and the requirements of ML models.

show abstract

Section: Explainable (Interpretable) Machine Learning For Genotype To...mentioning

confidence: 99%

Plant Genotype to Phenotype Prediction Using Machine Learning

et al. 2022

View full text Add to dashboard Cite

show abstract

“…To be precise, these rules were counterfactual and of the format "if the alcohol intake would have been 1 unit or less, the system would have advised a normal dose of insulin". Furthermore, Chromik et al (2021) found that users generalize from a collection of (contrasting, so including counterfactuals) local (Shapley) explanations, and typically do this incorrectly. That further supports that a pure case-by-case approach, where counterfactuals are presented but without overarching generalizations, doesn't truly explain the functioning of an algorithm.…”

Section: Counterfactuals Alonementioning

confidence: 99%

Defining Explanation and Explanatory Depth in XAI

Buijsman

2022

Minds & Machines

View full text Add to dashboard Cite

Explainable artificial intelligence (XAI) aims to help people understand black box algorithms, particularly of their outputs. But what are these explanations and when is one explanation better than another? The manipulationist definition of explanation from the philosophy of science offers good answers to these questions, holding that an explanation consists of a generalization that shows what happens in counterfactual cases. Furthermore, when it comes to explanatory depth this account holds that a generalization that has more abstract variables, is broader in scope and/or more accurate is better. By applying these definitions and contrasting them with alternative definitions in the XAI literature I hope to help clarify what a good explanation is for AI.

show abstract

“…Usages. Researchers have conducted user-studies on the use of certain explanations for certain stakeholders and data types [2,12,13,23,25,52]. Yet, no extensive work involves developers debugging computer vision models.…”

Section: Machine Learning Explainabilitymentioning

confidence: 99%

How can Explainability Methods be Used to Support Bug Identification in Computer Vision Models?

Balayn

Rikalo

Lofi

et al. 2022

CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

Deep learning models for image classifcation sufer from dangerous issues often discovered after deployment. The process of identifying bugs that cause these issues remains limited and understudied. Especially, explainability methods are often presented as obvious tools for bug identifcation. Yet, the current practice lacks an understanding of what kind of explanations can best support the diferent steps of the bug identifcation process, and how practitioners could interact with those explanations. Through a formative study and an iterative co-creation process, we build an interactive design probe providing various potentially relevant explainability functionalities, integrated into interfaces that allow for fexible workfows. Using the probe, we perform 18 user-studies with a diverse set of machine learning practitioners. Two-thirds of the practitioners engage in successful bug identifcation. They use multiple types of explanations, e.g. visual and textual ones, through non-standardized sequences of interactions including queries and exploration. Our results highlight the need for interactive, guiding, interfaces with diverse explanations, shedding light on future research directions. CCS CONCEPTS• Human-centered computing → User interface programming; Empirical studies in HCI ; • Computing methodologies → Computer vision; • Software and its engineering → Software testing and debugging.

show abstract

I Think I Get Your Point, AI! The Illusion of Explanatory Depth in Explainable AI

Cited by 76 publications

References 44 publications

Plant Genotype to Phenotype Prediction Using Machine Learning

Plant Genotype to Phenotype Prediction Using Machine Learning

Defining Explanation and Explanatory Depth in XAI

How can Explainability Methods be Used to Support Bug Identification in Computer Vision Models?

Contact Info

Product

Resources

About