Whole-Slide Mitosis Detection in H&E Breast Histology Using PHH3 as a Reference to Train Distilled Stain-Invariant Convolutional Networks

Abstract

Manual counting of mitotic tumor cells in tissue sections constitutes one of the strongest prognostic markers for breast cancer. This procedure, however, is time-consuming and error-prone. We developed a method to automatically detect mitotic figures in breast cancer tissue sections based on convolutional neural networks (CNNs). Application of CNNs to hematoxylin and eosin (H&E) stained histological tissue sections is hampered by: (1) noisy and expensive reference standards established by pathologists, (2) lack of generalization due to staining variation across laboratories, and (3) high computational requirements needed to process gigapixel whole-slide images (WSIs). In this paper, we present a method to train and evaluate CNNs to specifically solve these issues in the context of mitosis detection in breast cancer WSIs. First, by combining image analysis of mitotic activity in phosphohistone-H3 (PHH3) restained slides and registration, we built a reference standard for mitosis detection in entire H&E WSIs requiring minimal manual annotation effort. Second, we designed a data augmentation strategy that creates diverse and realistic H&E stain variations by modifying the hematoxylin and eosin color channels directly. Using it during training combined with network ensembling resulted in a stain invariant mitosis detector. Third, we applied knowledge distillation to reduce the computational requirements of the mitosis detection ensemble with a negligible loss of performance. The system was trained in a single-center cohort and evaluated in an independent multicenter cohort from The Cancer Genome Atlas on the three tasks of the Tumor Proliferation Assessment Challenge (TUPAC). We obtained a performance within the top-3 best methods for most of the tasks of the challenge.

Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathology

“…Common color augmentation techniques borrowed from Computer Vision include brightness, contrast and hue perturbations. Recently, researchers have proposed other approaches more tailored to mimic specific H&E stain variations by perturbing the images directly in the H&E color space (Tellez et al (2018)).…”

“…This intercenter stain variation hampers the performance of machine learning algorithms used for automatic WSI analysis. Algorithms that were trained with images originated from a single pathology laboratory often underperform when applied to images from a different center, including state-of-the-art methods based on convolutional neural networks (CNNs) (Goodfellow et al (2016); Bejnordi et al (2017); Bándi et al (2019); Tellez et al (2018); Veta et al (2018); Ciompi et al (2017); Sirinukunwattana et al (2017)). Existing solutions to reduce the generalization error in this setting can be categorized into two groups:…”

A deep learning-based model of normal histology

“…For each magnification, VGG-16, Inception-v3 and ResNet-50 networks -initialized on ImageNet -were trained on patches generated at the given magnification. During training, each patch was subjected to a random rotation and a random staining modulation according to the method described in (Tellez et al, 2018). This sample augmentation aims to render the classifiers robust to changes in orientation and variations in staining.…”

“…This means that in our approach each patch is used with the same rotation during each epoch. On the other hand, the stain modulation is performed during the training process (and thus differently also for the same patch across different epochs) according to the method described in (Tellez et al, 2018). Briefly, this method is based on first transforming from RGB color space to a color space consisting of one channel each for eosin, hematoxylin, and a third channel for the remaining color variation.…”

“…Finally, the modified patch is back-transformed into RGB color space. We experimented with different widths for the uniform distributions, and found that a parameter choice that doubles the variation reported in (Tellez et al, 2018) as 'typical' resulted in the best generalization ability, while leading to pronounced changes in the visual appearance of the patches. Note that even patches from a single slide will be oriented and stained independently from each other during the training process.…”