Explainability of Image Semantic Segmentation Through SHAP Values

Abstract

The introduction of Deep Neural Networks in high-level applications is significantly increasing. However, the understanding of such model decisions by humans is not straightforward and may limit their use for critical applications. In order to address this issue, recent research work has introduced explanation methods, typically for classification and captioning. Nevertheless, for some tasks, explainability methods need to be developed. This includes image segmentation that is an essential component for many high-level applications. In this paper, we propose a general workflow allowing for the adaptation of a state of the art explainability methods, especially SHAP, to image segmentation tasks. The approach allows for explanation of single pixels as well image areas. We show the relevance of the approach on a critical application such as oil slick pollution detection on the sea surface. We also show the applicability of the method on a more standard multimedia domain semantic segmentation task. The conducted experiments highlight the relevant features on which the models derive their local results and help identify general model behaviours.

An Explainable AI System for Medical Image Segmentation With Preserved Local Resolution: Mammogram Tumor Segmentation

“…not always improve the performance. In our case, the model hit the best similarity at class weights = [1,20] for the nonlesion and lesion classes. This emphasizes the necessity to fine-tune the class weights when considering the employment of the WCE loss in order to select the best-performing hyperparameters for the problem at hand.…”

“…A number of these revamped models have found their place in disease categorization networks, offering justification for predictions made by such opaque DL models [19]. Nevertheless, the quest to augment transparency in segmentation networks is still in its infancy [20], [21], although these too suffer from "black-box" frameworks. Additionally, post-model interpretative analysis can be pivotal for tech researchers to discern whether the model is pinpointing relevant patterns or merely over-optimizing based on training images' irrelevant attributes.…”

“…While much of the XAI work is centered around CNN classifications, only a few integrate this into semantic segmentation networks. In [20], the SHAP method is used for explanations in oil slick segmentation. For autonomous driving systems, [21] utilized the second-order neuron activation derivative to offer attention maps.…”

Comment on gmd-2024-60

“…SHAP can be implemented for semantic segmentation (e.g., Dardouillet et al, 2023), however, it computes relevance values for clusters of input grid points and not for individual input grid points. The same applies to LIME; moreover, to the best of our knowledge, we are not aware of implementations of LIME for semantic segmentation.…”