Validating Automatic Concept-Based Explanations for AI-Based Digital Histopathology

Abstract

Digital histopathology poses several challenges such as label noise, class imbalance, limited availability of labelled data, and several latent biases to deep learning, negatively influencing transparency, reproducibility, and classification performance. In particular, biases are well known to cause poor generalization. Proposed tools from explainable artificial intelligence (XAI), bias detection, and bias discovery suffer from technical challenges, complexity, unintuitive usage, inherent biases, or a semantic gap. A promising XAI method, not studied in the context of digital histopathology is automated concept-based explanation (ACE). It automatically extracts visual concepts from image data. Our objective is to evaluate ACE’s technical validity following design science principals and to compare it to Guided Gradient-weighted Class Activation Mapping (Grad-CAM), a conventional pixel-wise explanation method. To that extent, we created and studied five convolutional neural networks (CNNs) in four different skin cancer settings. Our results demonstrate that ACE is a valid tool for gaining insights into the decision process of histopathological CNNs that can go beyond explanations from the control method. ACE validly visualized a class sampling ratio bias, measurement bias, sampling bias, and class-correlated bias. Furthermore, the complementary use with Guided Grad-CAM offers several benefits. Finally, we propose practical solutions for several technical challenges. In contradiction to results from the literature, we noticed lower intuitiveness in some dermatopathology scenarios as compared to concept-based explanations on real-world images.

Artificial Intelligence in Melanoma Dermatopathology: A Review of Literature

“…The inability to understand how an AI system arrives at its decision has been termed the “black box problem.” Without explainability, can providers and researchers truly educate patients such that they can give informed consent? Sauter et al 52 investigated the utility of automated concept-based explanation in comprehending CNN image analysis wherein results showed that automated concept-based explanation could be an effective tool for analyzing CNN decision-making and enhancing researcher/physician diagnostic and treatment decisions while improving the transparency of CNNs. When mechanisms are properly understood and data sets are used responsibly, AI could optimize pathology workflows, enhance diagnostic accuracy, help the provider engage in prognostic prediction, and find individualized treatments.…”

Explainable AI in medical imaging: An overview for clinical practitioners – Beyond saliency-based XAI approaches

“…A significant advantage of ACE, compared to TCAV, is that it mitigates the need for human supervision, as manual labeling is not required. However, the method relies on the manual setting of parameters, which can affect the outcome, e.g., resulting in duplicated or mixed concepts [25].…”