Explaining Convolutional Neural Networks through Attribution-Based Input Sampling and Block-Wise Feature Aggregation

Abstract

As an emerging field in Machine Learning, Explainable AI (XAI) has been offering remarkable performance in interpreting the decisions made by Convolutional Neural Networks (CNNs). To achieve visual explanations for CNNs, methods based on class activation mapping and randomized input sampling have gained great popularity. However, the attribution methods based on these techniques provide lowerresolution and blurry explanation maps that limit their explanation power. To circumvent this issue, visualization based on various layers is sought. In this work, we collect visualization maps from multiple layers of the model based on an attribution-based input sampling technique and aggregate them to reach a fine-grained and complete explanation. We also propose a layer selection strategy that applies to the whole family of CNN-based models, based on which our extraction framework is applied to visualize the last layers of each convolutional block of the model. Moreover, we perform an empirical analysis of the efficacy of derived lower-level information to enhance the represented attributions. Comprehensive experiments conducted on shallow and deep models trained on natural and industrial datasets, using both groundtruth and model-truth based evaluation metrics validate our proposed algorithm by meeting or outperforming the stateof-the-art methods in terms of explanation ability and visual quality, demonstrating that our method shows stability regardless of the size of objects or instances to be explained.

Ada-SISE: Adaptive Semantic Input Sampling for Efficient Explanation of Convolutional Neural Networks

“…Two pre-trained models, a shallow VGG16 (with a test accuracy of 87.18%) and a residual ResNet-50 network (with 87.96% test accuracy), are directly loaded from the TorchRay library [10] to replicate the original experimentation setup. As it was reported in [5] that SISE meets or outperforms most of the state-of-the-art XAI methods like Grad-CAM [14], RISE [10] and Score-CAM [16], we restrict our comparisons only with Extremal Perturbation [11] (as it is one of the sophisticated perturbation-based methods) and SISE. 1.…”

“…Table 1 shows the benchmark evaluation of Ada-SISE concerning various metrics and their execution time. As the depicted results are achieved through the same experimental setup as SISE paper, the readers can refer to [5] to infer further head-to-head comparison of Ada-SISE with other state-ofthe-art methods. Energy-Based Pointing Game (EBPG) [16] and Bbox [19] use the ground-truth annotations available to determine the precision of an XAI algorithm.…”

“…In this work, we study such a recent post-hoc, local, and model-specific XAI algorithm -Semantic Input Sampling for Explanation (SISE) [5] developed for image classification tasks. Building on this method, we propose a way to improve its run-time while retaining its overall performance without compromising the visual explanation's quality.…”

Logic Rule Guided Attribution with Dynamic Ablation

“…Ablation-CAM (Ramaswamy et al 2020) and Score-CAM (Wang et al 2020) have been developed to overcome these drawbacks. Despite the strength of the CAMbased methods in capturing the features extracted in CNNs, the lack of localization information in the coarse high-level feature maps limits such methods' performance by producing blurry explanations (Sattarzadeh et al 2020).…”

SeCAM: Tightly Accelerate the Image Explanation via Region-Based Segmentation

“…Those superpixels come from a previous perturbation step. With visualization algorithms like CAM (Class Activation Mapping) [6], SISE (Semantic Input Sampling for Explanations) [7], Saliency Map [8], and so on, the output will show the user a heatmap on the original image.…”