The Effect of Within-Bag Sampling on End-to-End Multiple Instance Learning

Koriakina, Nadezhda; Sladoje, Nataša; Lindblad, Joakim

doi:10.1109/ispa52656.2021.9552170

Cited by 5 publications

(13 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, for TUPAC16, where image features associated with tumor proliferation scores are more uniformly distributed throughout the slides, sampling only 8 patches at training time is sufficient for the network to achieve optimal performance. Moreover, these results emphasize the importance of carefully selecting the number of instances for each dataset to achieve high performance with the assistance of random sampling, as noted in [4,10]. However, we observed that random sampling enhanced the model's interpretability on CAMELYON16, as it prevents overfitting on this small dataset.…”

Section: Discussionsupporting

confidence: 52%

“…The utilization of random sampling during training of deep networks in combination with MIL has been previously explored in tasks involving both natural images [10] and medical images [11], including histopathology [3,4]. Although seemingly straightforward, this technique allows for end-to-end training of MIL with larger bags.…”

Section: Mil For Wsi Classificationmentioning

confidence: 99%

See 1 more Smart Citation

Multiple instance learning with random sampling for whole slide image classification

Keshvarikhojasteh,

Pluim,

Veta

2024

Medical Imaging 2024: Digital and Computational Pathology

View full text Add to dashboard Cite

In computational pathology, random sampling of patches during training of Multiple Instance Learning (MIL) methods is computationally efficient and serves as a regularization strategy. Despite its promising benefits, questions concerning performance trends for varying sample sizes and its influence on model interpretability remain. Addressing these, we reach an optimal performance enhancement of 1.7% using thirty percent of patches on the CAMELYON16 dataset, and 3.7% with only eight samples on the TUPAC16 dataset. We also find interpretability effects are strongly datasetdependent, with interpretability impacted on CAMELYON16, while remaining unaffected on TUPAC16. This reinforces that both the performance and interpretability relationships with sampling are closely task-specific. End-to-end training with 1024 samples reveals improvements across both datasets compared to pre-extracted features, further highlighting the potential of this efficient approach.

show abstract

Section: Discussionsupporting

confidence: 52%

Section: Mil For Wsi Classificationmentioning

confidence: 99%

Multiple instance learning with random sampling for whole slide image classification

Keshvarikhojasteh,

Pluim,

Veta

2024

Medical Imaging 2024: Digital and Computational Pathology

View full text Add to dashboard Cite

show abstract

“…Other MIL approaches tackle this problem using, for example, self-supervision [8], selection of instances in a bag during training [10] or features learned by models pretrained on ImageNet [7,11]. In a previous study [14], we observed that ABMIL can work well with within-bag sampling, benefiting from end-to-end learning, while overcoming memory requirements imposed by WSI. Similarly to ABMIL with sampling, EWSLF [12] is trained end-to-end and offers reduced computational cost of handling large WSI data.…”

Section: Background and Related Workmentioning

confidence: 99%

Deep multiple instance learning versus conventional deep single instance learning for interpretable oral cancer detection

Koriakina,

Sladoje,

Bašić

et al. 2024

PLoS ONE

Self Cite

View full text Add to dashboard Cite

The current medical standard for setting an oral cancer (OC) diagnosis is histological examination of a tissue sample taken from the oral cavity. This process is time-consuming and more invasive than an alternative approach of acquiring a brush sample followed by cytological analysis. Using a microscope, skilled cytotechnologists are able to detect changes due to malignancy; however, introducing this approach into clinical routine is associated with challenges such as a lack of resources and experts. To design a trustworthy OC detection system that can assist cytotechnologists, we are interested in deep learning based methods that can reliably detect cancer, given only per-patient labels (thereby minimizing annotation bias), and also provide information regarding which cells are most relevant for the diagnosis (thereby enabling supervision and understanding). In this study, we perform a comparison of two approaches suitable for OC detection and interpretation: (i) conventional single instance learning (SIL) approach and (ii) a modern multiple instance learning (MIL) method. To facilitate systematic evaluation of the considered approaches, we, in addition to a real OC dataset with patient-level ground truth annotations, also introduce a synthetic dataset—PAP-QMNIST. This dataset shares several properties of OC data, such as image size and large and varied number of instances per bag, and may therefore act as a proxy model of a real OC dataset, while, in contrast to OC data, it offers reliable per-instance ground truth, as defined by design. PAP-QMNIST has the additional advantage of being visually interpretable for non-experts, which simplifies analysis of the behavior of methods. For both OC and PAP-QMNIST data, we evaluate performance of the methods utilizing three different neural network architectures. Our study indicates, somewhat surprisingly, that on both synthetic and real data, the performance of the SIL approach is better or equal to the performance of the MIL approach. Visual examination by cytotechnologist indicates that the methods manage to identify cells which deviate from normality, including malignant cells as well as those suspicious for dysplasia. We share the code as open source.

show abstract

“…10 Within-bag sampling has previously been integrated with ABMIL by splitting each bag into a group of minibags -overlapping subsets of the original bag. 11 Each mini-bag is classified through an ABMIL model which has been trained on mini-bags, and the bag classifications are given by the majority vote of the mini-bags. Mini-bag processing reduces memory requirements, but the duplication of instances across multiple mini-bags is likely to increase inference time.…”

Section: Within-bag Instance Samplingmentioning

confidence: 99%

“…Further, as the key instance detection is based upon the ABMIL attention weights, all instances need to be passed through the feature extraction part of the network, which has a high computational burden. Subsequent work 12 showed this approach to be less accurate than conventional single instance learning for cytological data, but it has not been evaluated for whole slide histopathological data, where single instance learning is not practical due to the large image sizes. Some MIL sampling approaches use patch classification probabilities rather than attention scores.…”

Section: Within-bag Instance Samplingmentioning

confidence: 99%

Efficient subtyping of ovarian cancer histopathology whole slide images using active sampling in multiple instance learning

Breen¹,

Allen

Zucker

et al. 2023

Medical Imaging 2023: Digital and Computational Pathology

View full text Add to dashboard Cite

Weakly-supervised classification of histopathology slides is a computationally intensive task, with a typical whole slide image (WSI) containing billions of pixels to process. We propose Discriminative Region Active Sampling for Multiple Instance Learning (DRAS-MIL), a computationally efficient slide classification method using attention scores to focus sampling on highly discriminative regions. We apply this to the diagnosis of ovarian cancer histological subtypes, which is an essential part of the patient care pathway as different subtypes have different genetic and molecular profiles, treatment options, and patient outcomes. We use a dataset of 714 WSIs acquired from 147 epithelial ovarian cancer patients at Leeds Teaching Hospitals NHS Trust to distinguish the most common subtype, high-grade serous carcinoma, from the other four subtypes (low-grade serous, endometrioid, clear cell, and mucinous carcinomas) combined. We demonstrate that DRAS-MIL can achieve similar classification performance to exhaustive slide analysis, with a 3-fold cross-validated AUC of 0.8679 compared to 0.8781 with standard attention-based MIL classification. Our approach uses at most 18% as much memory as the standard approach, while taking 33% of the time when evaluating on a GPU and only 14% on a CPU alone. Reducing prediction time and memory requirements may benefit clinical deployment and the democratisation of AI, reducing the extent to which computational hardware limits end-user adoption.

show abstract

The Effect of Within-Bag Sampling on End-to-End Multiple Instance Learning

Cited by 5 publications

References 11 publications

Multiple instance learning with random sampling for whole slide image classification

Multiple instance learning with random sampling for whole slide image classification

Deep multiple instance learning versus conventional deep single instance learning for interpretable oral cancer detection

Efficient subtyping of ovarian cancer histopathology whole slide images using active sampling in multiple instance learning

Contact Info

Product

Resources

About