Integration of Local and Global Features Explanation with Global Rules Extraction and Generation Tools

Abstract

Widely used in a growing number of domains, Deep Learning predictors are achieving remarkable results. However, the lack of transparency (i.e., opacity) of their inner mechanisms has raised trust and employability concerns. Nevertheless, several approaches fostering models of interpretability and explainability have been developed in the last decade. This paper combines approaches for local feature explanation (i.e., Contextual Importance and Utility -CIU) and global feature explanation (i.e., Explainable Layers) with a rule extraction system, namely ECLAIRE. The proposed pipeline has been tested in four scenarios employing a breast cancer diagnosis dataset. The results show improvements such as the production of more human-interpretable rules and adherence of the produced rules with the original model.

A DEXiRE for Extracting Propositional Rules from Neural Networks via Binarization

“…Accuracy of a predictor is a measure of the quality of predictions, which compares the prediction of a model with the ground-truth labels, counting the fraction of correct predictions over the total number of predictions (Equation (3)). Accuracy is measured using different metrics like accuracy-score, F1-score, precision, recall, and other performance metrics, evaluating the predictions against the ground-truth labels on supervised datasets [60,61].…”

“…Fidelity is a metric that compares the predictions from the original black-box model ( ŷ) and the predictions from an interpretable model ( ŷrs ), measuring how reliable the explanations are in reflecting the underlying model's behavior (Equation ( 4)). Fidelity is measured in terms of accuracy, F1-score, and other similarity measures, using the predictions of the black-box model as the ground truth [60][61][62]. Fidelity = Accuracy( ŷ, ŷrs )…”

“…Two example activation patterns are shown in Figures 4 and 5 S7: Features' rule generation: Binary activation of the first hidden layer (h b 0 ) is used as the target label to induce features' rule set. Explainable layers (ExpL) are hidden layers that are able, through their weights and activation, to explain the underlying neuron behavior [60]. ExpL layers can learn an activation threshold t i for the features producing partitions in the feature space, in the form (x i ≥ t i ), according to the class label.…”

“…On the one hand, for binary classification (SC1), S7 consists of learning the initial rule set (the rule set between the input features and the first hidden layer) using explainable layers (ExpL). ExpL is an additional layer added to the original mode that can learn the activation threshold that splits the decision region between two classes [60]. On the other hand, multiclass classification (SC2) concerns three or more target categories.…”

“…After a fine-tuning training with the rest of the layers frozen, the weights of ExpL codify a threshold value (v i ) per feature (x i ) that splits the input space in two, indicating the point of maximum information gain per feature. Using the learned threshold values is possible to induce simple rules per feature (x i ) in the form of i f (x i ≥ v i ) then class 1; else class 0, combining these simple rules into the intermediate rule set the final rule set is obtained [60]. ExpL layers are suitable for binary classification tasks, given their ability to split the space into binary buckets with complementary rules.…”