Kernel Self-Attention in Deep Multiple Instance Learning

Rymarczyk, Dawid; Borowa, Adriana; Tabor, Jacek; Zieliński, Bartosz

doi:10.48550/arxiv.2005.12991

Cited by 4 publications

(5 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Image classification network. To aggregate patches' representations of an image, we apply attention-based multiple instance learning, AbMIL [10,17], a type of weighted average pooling, where the neural network determines the weights of representations. More formally, if…”

Section: Methodsmentioning

confidence: 99%

Classifying bacteria clones using attention-based deep multiple instance learning interpreted by persistence homology

Borowa,

Rymarczyk,

Ochońska

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

In this work, we analyze if it is possible to distinguish between different clones of the same bacteria species (Klebisiella pneumoniae) based only on microscopic images. It is a challenging task, previously considered impossible due to the high clones' similarity. For this purpose, we apply a multi-step algorithm with attention-based multiple instance learning. Except for obtaining accuracy at the level of 0.9, we introduce extensive interpretability based on CellProfiler and persistence homology, increasing the understandability and trust in the model.

show abstract

Section: Methodsmentioning

confidence: 99%

Classifying bacteria clones using attention-based deep multiple instance learning interpreted by persistence homology

Borowa,

Rymarczyk,

Ochońska

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…3, S1). These are further cross-correlated in the self-attention 26 module, to account for dependencies across the image and to create context-aware embeddings from every patch. The subsequent attention-based aggregation module 27 assigns a score of relative importance to every patch and combines them into a single representation.…”

Section: Pcai Design Rationalementioning

confidence: 99%

Robust, credible, and interpretable AI-based histopathological prostate cancer grading

Westhaeusser,

Fuhlert,

Dietrich

et al. 2024

Preprint

View full text Add to dashboard Cite

SummaryBackgroundProstate cancer (PCa) is among the most common cancers in men and its diagnosis requires the histopathological evaluation of biopsies by human experts. While several recent artificial intelligence-based (AI) approaches have reached human expert-level PCa grading, they often display significantly reduced performance on external datasets. This reduced performance can be caused by variations in sample preparation, for instance the staining protocol, section thickness, or scanner used. Another limiting factor of contemporary AI-based PCa grading is the prediction of ISUP grades, which leads to the perpetuation of human annotation errors.MethodsWe developed theprostatecanceraggressiveness index (PCAI), an AI-based PCa detection and grading framework that is trained on objective patient outcome, rather than subjective ISUP grades. We designed PCAI as a clinical application, containing algorithmic modules that offer robustness to data variation, medical interpretability, and a measure of prediction confidence. To train and evaluate PCAI, we generated a multicentric, retrospective, observational trial consisting of six cohorts with 25,591 patients, 83,864 images, and 5 years of median follow-up from 5 different centers and 3 countries. This includes a high-variance dataset of 8,157 patients and 28,236 images with variations in sample thickness, staining protocol, and scanner, allowing for the systematic evaluation and optimization of model robustness to data variation. The performance of PCAI was assessed on three external test cohorts from two countries, comprising 2,255 patients and 9,437 images.FindingsUsing our high-variance datasets, we show how differences in sample processing, particularly slide thickness and staining time, significantly reduce the performance of AI-based PCa grading by up to 6.2 percentage points in the concordance index (C-index). We show how a select set of algorithmic improvements, including domain adversarial training, conferred robustness to data variation, interpretability, and a measure of credibility to PCAI. These changes lead to significant prediction improvement across two biopsy cohorts and one TMA cohort, systematically exceeding expert ISUP grading in C-index and AUROC by up to 22 percentage points.InterpretationData variation poses serious risks for AI-based histopathological PCa grading, even when models are trained on large datasets. Algorithmic improvements for model robustness, interpretability, credibility, and training on high-variance data as well as outcome-based severity prediction gives rise to robust models with above ISUP-level PCa grading performance.

show abstract

“…High recall is especially important for the case of cancer detection since fail to detect the presence of cancer can lead to sever consequence. We ran the experiment 5 times each with a 10fold cross-validation and summarized the results compared to reports in [8] and [14] in Tab. 6 and Tab.…”

Section: Histopathological Datasetmentioning

confidence: 99%

“…The attention score reflect how much an instance is likely to be the key instance that trigger the bag classifier. Later on, [14,18,12] proposed to use self-attention mechanism [17] to further consider the the dependencies between instance embeddings. However, computing the self-attention matrix across all instance embeddings in a bag is computationally complex and might yield redundant information that does not contributes useful supervising signal.…”

Section: Introductionmentioning

confidence: 99%

Dual-stream Maximum Self-attention Multi-instance Learning

Li,

Eliceiri

2020

Preprint

View full text Add to dashboard Cite

Multi-instance learning (MIL) is a form of weakly supervised learning where a single class label is assigned to a bag of instances while the instance-level labels are not available. Training classifiers to accurately determine the bag label and instance labels is a challenging but critical task in many practical scenarios, such as computational histopathology. Recently, MIL models fully parameterized by neural networks have become popular due to the high flexibility and superior performance. Most of these models rely on attention mechanisms that assign attention scores across the instance embeddings in a bag and produce the bag embedding using an aggregation operator. In this paper, we proposed a dual-stream maximum selfattention MIL model (DSMIL) parameterized by neural networks. The first stream deploys a simple MIL max-pooling while the top-activated instance embedding is determined and used to obtain self-attention scores across instance embeddings in the second stream. Different from most of the previous methods, the proposed model jointly learns an instance classifier and a bag classifier based on the same instance embeddings. The experiments results show that our method achieves superior performance compared to the best MIL methods and demonstrates state-of-the-art performance on benchmark MIL datasets.

show abstract

Kernel Self-Attention in Deep Multiple Instance Learning

Cited by 4 publications

References 20 publications

Classifying bacteria clones using attention-based deep multiple instance learning interpreted by persistence homology

Classifying bacteria clones using attention-based deep multiple instance learning interpreted by persistence homology

Robust, credible, and interpretable AI-based histopathological prostate cancer grading

Dual-stream Maximum Self-attention Multi-instance Learning

Contact Info

Product

Resources

About