Factual and Counterfactual Explanations for Black Box Decision Making

Abstract

The rise of sophisticated machine learning models has brought accurate but obscure decision systems, which hide their logic, thus undermining transparency, trust, and the adoption of AI in socially sensitive and safety-critical contexts. We introduce a local rule-based explanation method providing faithful explanations of the decision made by a black-box classifier on a specific instance. The proposed method first learns an interpretable, local classifier on a synthetic neighborhood of the instance under investigation, generated by a genetic algorithm. Then it derives from the interpretable classifier an explanation consisting of a decision rule, explaining the factual reasons of the decision, and a set of counterfactuals, suggesting the changes in the instance features that would lead to a different outcome. Experimental results show that the proposed method outperforms existing approaches in terms of the quality of the explanations and of the accuracy in mimicking the black-box.

Capturing Users’ Reality: A Novel Approach to Generate Coherent Counterfactual Explanations

“…XAI literature proposes two main classes of methods to identify the foil. The first class locally approximates the AI system with a simpler model from which a foil is extracted [20]. For example, a decision tree is used to approximate the AI system's outputs in the vicinity of the fact to derive a foil that lies close to the fact with respect to the decision tree's structure [21].…”

Explaining Sentiment Classification with Synthetic Exemplars and Counter-Exemplars

“…xspells , falls into the category of local , model-agnostic methods which originated with [ 33 ] and extended along diverse directions by [ 12 ] and by [ 14 , 16 ]. Well known model-agnostic local explanation methods able to also work on textual data include lime , anchor and shap .…”

“…Rules can also be used to provide counter-factuals , namely alternative conditions, not met by x , that would determine a different answer by the black box [ 4 ]. In our approach, we will build on lore [ 14 ], a local explainer for tabular data that learns a decision tree from a given neighborhood Z of the instance to explain. Such a tree is a surrogate model of the black box, i.e., it is trained to reproduce the decisions of the black box.…”

“…xspells generates a set Z of n instances in the latent feature space for a given z . The neighborhood generation function (line 2) can be implemented by adopting several different strategies, ranging from a purely random approach like in lime [ 33 ], to using a given distribution and a genetic algorithm maximizing a fitness function like in lore [ 14 ]. xspells adopts a random generation of latent synthetic instances by relying on the fact that the encoder maps uniformly the data distribution over the latent space.…”

“…Given Z and , xspells builds a latent decision tree (line 5) acting as a local surrogate of the black box, i.e., being able to locally mime the behavior of b . xspells adopts decision tree because decision rules can be derived from a root-to-leaf path [ 14 ]. Indeed, the premise p of the rule is the conjunction of the split conditions from the root to the leaf of the tree that is followed by features in z .…”