Targeting tumor heterogeneity: multiplex-detection-based multiple instance learning for whole slide image classification

Abstract

Motivation Multiple instance learning (MIL) is a powerful technique to classify whole slide images (WSIs) for diagnostic pathology. The key challenge of MIL on WSI classification is to discover the critical instances that trigger the bag label. However, tumor heterogeneity significantly hinders the algorithm’s performance. Results Here, we propose a novel multiplex-detection-based multiple instance learning (MDMIL) which targets tumor heterogeneity by multiplex detection strategy and feature constraints among samples. Specifically, the internal query (IQ) generated after the probability distribution analysis and the variational query (VQ) optimized throughout the training process are utilized to detect potential instances in the form of internal and external assistance, respectively. The multiplex detection strategy significantly improves the instance-mining capacity of the deep neural network. Meanwhile, a memory-based contrastive loss is proposed to reach consistency on various phenotypes in the feature space. The novel network and loss function jointly achieve high robustness towards tumor heterogeneity. We conduct experiments on three computational pathology datasets, e.g., CAMELYON16, TCGA-NSCLC, and TCGA-RCC. Benchmarking experiments on the three datasets illustrate that our proposed MDMIL approach achieves superior performance over several existing state-of-the-art methods. Availability MDMIL is available for academic purposes at https://github.com/ZacharyWang-007/MDMIL. Supplementary information Supplementary data are available at Bioinformatics online.

Deep Learning for Lung Cancer Diagnosis, Prognosis and Prediction Using Histological and Cytological Images: A Systematic Review

“…Their model performed with good classification accuracy, while the accuracy was slightly increased in the external validation cohorts when tissue microarrays (TMAs) were selected (accuracy rates of 0.78 in biopsies versus 0.82 in TMAs). Finally, two recent studies performed a binary classification between ADC and SCC using over 900 WSIs from TCGA and achieving an AUC of over 0.90 [34,35] (Table 2).…”

“…General approaches of weakly supervised learning in histopathological images have been proposed, employing VGG-16 [25], EM-CNN [52], EfficientNet-B3 [20], and ResNet [80]. Furthermore, most of the presented studies in this category employ MIL [34,85], which is a weakly supervised learning technique that groups data points into bags. Each bag is labeled with the class by the instance count of that particular class.…”

Convolutional Sparse Coding Multiple Instance Learning for Whole Slide Image Classification

“…The potential sub-optimality of Specifically, the limited representation of positive instances within a positive bag can lead to the MIL aggregator learning an inaccurate mapping between embeddings and labels. Additionally, the limited supervisory signal poses a hindrance to the MIL aggregator's ability to capture correlations among instances [7,8,10].…”

“…Prior endeavors at MIL addressed these two obstacles individually. The initial category of approaches centered on enhancing the feature embeddings that were extracted through the use of self-supervised pretraining [7,10,11,12].…”

“…Better sparse instance feature embeddings that can model the invariance of the same type of biological tissues would also ease the duty of the MIL aggregator, therefore there is a strong relationship between these two types of approaches. Sparse feature embeddings, which are a lowdimensional version of the feature extractor's initial instance embeddings, are also advantageous to WSI representation since the high-dimensional WSI representation is located on a low-dimensional manifold, according to empirical evidence [10,12].…”