Explainability and causability in digital pathology

Abstract

The current move towards digital pathology enables pathologists to use artificial intelligence (AI)‐based computer programmes for the advanced analysis of whole slide images. However, currently, the best‐performing AI algorithms for image analysis are deemed black boxes since it remains – even to their developers – often unclear why the algorithm delivered a particular result. Especially in medicine, a better understanding of algorithmic decisions is essential to avoid mistakes and adverse effects on patients. This review article aims to provide medical experts with insights on the issue of explainability in digital pathology. A short introduction to the relevant underlying core concepts of machine learning shall nurture the reader's understanding of why explainability is a specific issue in this field. Addressing this issue of explainability, the rapidly evolving research field of explainable AI (XAI) has developed many techniques and methods to make black‐box machine‐learning systems more transparent. These XAI methods are a first step towards making black‐box AI systems understandable by humans. However, we argue that an explanation interface must complement these explainable models to make their results useful to human stakeholders and achieve a high level of causability, i.e. a high level of causal understanding by the user. This is especially relevant in the medical field since explainability and causability play a crucial role also for compliance with regulatory requirements. We conclude by promoting the need for novel user interfaces for AI applications in pathology, which enable contextual understanding and allow the medical expert to ask interactive ‘what‐if’‐questions. In pathology, such user interfaces will not only be important to achieve a high level of causability. They will also be crucial for keeping the human‐in‐the‐loop and bringing medical experts' experience and conceptual knowledge to AI processes.

Computational pathology-based weakly supervised prediction model for MGMT promoter methylation status in glioblastoma

“…We do not know what features in the pathological sections make the model final, which is a problem for clinicians. To solve the problem of explainability of models, the development of explainable computational pathology combined with explainable AI in recent years may be an important direction for future research ( 39 ).…”

Seeing the random forest through the decision trees. Supporting learning health systems from histopathology with machine learning models: Challenges and opportunities

“…And, as there is still a large gap between ML models' pattern recognition and human-level concept learning, 118 the ability to understand all the concepts included in diagnostic/classification criteria may be beyond current ML models' capabilities. Noteworthily, concept-based explainable methods 119,120 could facilitate a rigorous assessment of this capability by pathologists' in the future.…”

A Comprehensive Survey of ArtificialIntelligence in Precision Healthcare:Shedding Light on Interpretability

“…The importance of proving the scientific validity, analytical capability, and clinical effectiveness of AI applications in compliance with legal requirements is covered. The necessity for successful explanations that reach a certain level of causal knowledge in a particular context is highlighted by the introduction of the idea of causability as a metric of the usefulness of explanations in human AI interaction [75].…”