Classification of colorectal tissue images from high throughput tissue microarrays by ensemble deep learning methods

Abstract

Tissue microarray (TMA) core images are a treasure trove for artificial intelligence applications. However, a common problem of TMAs is multiple sectioning, which can change the content of the intended tissue core and requires re-labelling. Here, we investigate different ensemble methods for colorectal tissue classification using high-throughput TMAs. Hematoxylin and Eosin (H&E) core images of 0.6 mm or 1.0 mm diameter from three international cohorts were extracted from 54 digital slides (n = 15,150 cores). After TMA core extraction and color enhancement, five different flows of independent and ensemble deep learning were applied. Training and testing data with 2144 and 13,006 cores included three classes: tumor, normal or “other” tissue. Ground-truth data were collected from 30 ngTMA slides (n = 8689 cores). A test augmentation is applied to reduce the uncertain prediction. Predictive accuracy of the best method, namely Soft Voting Ensemble of one VGG and one CapsNet models was 0.982, 0.947 and 0.939 for normal, “other” and tumor, which outperformed to independent or ensemble learning with one base-estimator. Our high-accuracy algorithm for colorectal tissue classification in high-throughput TMAs is amenable to images from different institutions, core sizes and stain intensity. It helps to reduce error in TMA core evaluations with previously given labels.

Image-based assessment of extracellular mucin-to-tumor area predicts consensus molecular subtypes (CMS) in colorectal cancer

“…An expert pathologist manually selects a few example patches of tissues of interest by using some stand-alone software, such as QuPath 16 and ASAP (https://github.com/ computationalpathologygroup/ASAP). On the other hand, existing automatic extraction tools using deep learning to classify and locate the tissue of interest can be applied [17][18][19] . Especially in our previous work 19 , we proposed a system for tissue detection in WSI based on an ensemble learning method with two raters, a VGG 20 and a CapsuleNet 21 .…”

“…On the other hand, existing automatic extraction tools using deep learning to classify and locate the tissue of interest can be applied [17][18][19] . Especially in our previous work 19 , we proposed a system for tissue detection in WSI based on an ensemble learning method with two raters, a VGG 20 and a CapsuleNet 21 . Some additional examples of the tissue segmentation output are shown in Supplementary Fig.…”

COVID-CCD-Net: COVID-19 and colon cancer diagnosis system with optimized CNN hyperparameters using gradient-based optimizer

“…As can be seen in various studies in the existing literature, the number of studies dealing with automatic diagnosis of colon cancer in TMAs is limited. Nguyen et al [ 55 ] analyzed different ensemble approaches for colorectal tissue classification using highly efficient TMAs and proposed an ensemble deep learning–based approach with two different neural network architectures called VGG16 and CapsNet. Thanks to this approach, they classified colorectal tissues in highly efficient TMAs into three different categories, namely tumor, normal, and stroma/others.…”

Evaluating Generic Auto-ML Tools for Computational Pathology

“…On the other hand, this discrepancy indicates that the underlying (diagnostic) tasks are robust with respect to misclassi cations of part of the data for separate cases. One way to mitigate or pro t from this variability is to generate multiple CNNs and use them as an ensemble [85,86,87,88,89], either at the tile level before aggregating information per slide, or at the slide level-a detailed investigation of which is also beyond the scope of this study.…”