CoCoX: Generating Conceptual and Counterfactual Explanations via Fault-Lines

Abstract

We present CoCoX (short for Conceptual and Counterfactual Explanations), a model for explaining decisions made by a deep convolutional neural network (CNN). In Cognitive Psychology, the factors (or semantic-level features) that humans zoom in on when they imagine an alternative to a model prediction are often referred to as fault-lines. Motivated by this, our CoCoX model explains decisions made by a CNN using fault-lines. Specifically, given an input image I for which a CNN classification model M predicts class cpred, our fault-line based explanation identifies the minimal semantic-level features (e.g., stripes on zebra, pointed ears of dog), referred to as explainable concepts, that need to be added to or deleted from I in order to alter the classification category of I by M to another specified class calt. We argue that, due to the conceptual and counterfactual nature of fault-lines, our CoCoX explanations are practical and more natural for both expert and non-expert users to understand the internal workings of complex deep learning models. Extensive quantitative and qualitative experiments verify our hypotheses, showing that CoCoX significantly outperforms the state-of-the-art explainable AI models. Our implementation is available at https://github.com/arjunakula/CoCoX

Contrastive explanation: a structural-model approach

“…(Buchanan & Shortliffe, 1984), which is providing a foil for the fact. More recently, there has been a keen interest in answering why-not questions for many different sub-fields of artificial intelligence, including machine learning classification (Dhurandhar et al" 2018;Mothilal et al" 2020), belief-desireintention agents (Winikoff, 2017), reinforcement learning (Madumal et al" 2020;Waa et al" 2018), classical planning (Krarup et al" 2019;Sreedharan et al" 2018), and image classification (Akula et al" 2020), to cite just a few papers.…”

“…This leads to algorithms that can be useful, but terminology and solutions that are not aligned. For example, Dhurandhar et al (2018) use the term pertinent negatives/positives to refer to foils, while Akula et al (2020) use the term fault lines, and Krarup et al (2019) use foil.…”

Human In Command Machine Learning

“…In that way, a taxonomy the ML system can use to create causal attributions can be created and expanded towards a semantic level, using concepts identified and named by human users during usage. This borders recent work by Akula et al (2020) that identifies concepts like, 'stripedness', 'beard', 'horn', et cetera that can be used to create counterfactual explanations building on minimal concept changes that changes the classification, from for example goat to sheep. Our problem differs in that the concepts identified can be very subtle, relative and can even be, in a system that enhances human capabilities, not visible to the naked eye.…”

“…In ML there exists several methods both for lifting out causal attributes and work that aims to name and identify these attributes as concepts using previously defined concepts (Ghorbani et al, 2019;Amershi et al, 2009;Koh et al, 2020;Gonzalez-Garcia et al, 2018;Kornblith et al, 2019;Samek et al, 2020;Bengio et al, 2013;Akula et al, 2020). There exists, to our knowledge less work that focuses on finding and naming concepts during usage (Ghorbani et al, 2019).…”

“…There is a growing research interest focused on explaining these blackboxed decisions by summarising reasons for a decision Akula et al, 2020;Bengio et al, 2013;Ghorbani et al, 2019;Gonzalez-Garcia et al, 2018). A large quantity of the current work focus on creating causal attributions related to the trained ML model as such, and less focus is on supporting user's understanding of the ML system's capabilities Lu et al, 2019).…”