ProtoMIL: Multiple Instance Learning with Prototypical Parts for Whole-Slide Image Classification

Abstract

Multiple Instance Learning (MIL) gains popularity in many real-life machine learning applications due to its weakly supervised nature. However, the corresponding effort on explaining MIL lags behind, and it is usually limited to presenting instances of a bag that are crucial for a particular prediction. In this paper, we fill this gap by introducing ProtoMIL, a novel self-explainable MIL method inspired by the case-based reasoning process that operates on visual prototypes. Thanks to incorporating prototypical features into objects description, ProtoMIL unprecedentedly joins the model accuracy and fine-grained interpretability, which we present with the experiments on five recognized MIL datasets.

Multi-Attention Multiple Instance Learning

“…Elements of the explanation methods [40,41,42,43] were used to provide interpretability of the MIL models [22].…”

“…Starting from the attention-based MIL [21] and following this work, several interesting models using the attention mechanism have been developed. They, for example, include DeepAttnMISL (Deep Attention Multiple Instance Survival Learning) [12], MHAttnSurv (Multi-Head Attention for Survival Prediction) [23], ProtoMIL (Multiple Instance Learning with Prototypical Parts) [22], SA-AbMILP (Self-Attention Attention-based MIL Pooling) [46], the loss-attention MIL (the instance weights are calculated based on the loss function) [24], DSMIL (Dual-stream Multiple Instance Learning) [27] MILL (Multiple Instance Learning-based Landslide classification) [25], AbDMIL [29]. There are other MIL methods using the attention mechanism, which can be found in [26,47,28].…”

“…An attention-based MIL was proposed in [21] where the linear regression model was replaced by a one-layer neural network with single output. Following this work, several methods of MIL based on the attention mechanism have been developed, including Deep Attention Multiple Instance Survival Learning [12], Pro-toMIL [22], MHAttnSurv [23], the loss-attention MIL [24], MILL [25]. There are other MIL methods using the attention mechanism, which can be found in [26,27,28].…”

Explainable AI and interpretable computer vision : from oversight to insight

“…Most of these approaches are developed for fine-grained natural image recognition, including recognising bird species and car types. Only a few works apply part-prototype models to medical images: ProtoPNet [CLT+19] is also applied to chest X-rays [SY21] and MRI scans [MCT+21], ProtoMIL [RKK+21] is developed for histology slide classification and [BST+21] adapted ProtoPNet for mammography by including a loss based on fine-grained expert image annotations.…”