Empirical comparison of color normalization methods for epithelial-stromal classification in H and E images

Abstract

Context:Color normalization techniques for histology have not been empirically tested for their utility for computational pathology pipelines.Aims:We compared two contemporary techniques for achieving a common intermediate goal – epithelial-stromal classification.Settings and Design:Expert-annotated regions of epithelium and stroma were treated as ground truth for comparing classifiers on original and color-normalized images.Materials and Methods:Epithelial and stromal regions were annotated on thirty diverse-appearing H and E stained prostate cancer tissue microarray cores. Corresponding sets of thirty images each were generated using the two color normalization techniques. Color metrics were compared for original and color-normalized images. Separate epithelial-stromal classifiers were trained and compared on test images. Main analyses were conducted using a multiresolution segmentation (MRS) approach; comparative analyses using two other classification approaches (convolutional neural network [CNN], Wndchrm) were also performed.Statistical Analysis:For the main MRS method, which relied on classification of super-pixels, the number of variables used was reduced using backward elimination without compromising accuracy, and test - area under the curves (AUCs) were compared for original and normalized images. For CNN and Wndchrm, pixel classification test-AUCs were compared.Results:Khan method reduced color saturation while Vahadane reduced hue variance. Super-pixel-level test-AUC for MRS was 0.010–0.025 (95% confidence interval limits ± 0.004) higher for the two normalized image sets compared to the original in the 10–80 variable range. Improvement in pixel classification accuracy was also observed for CNN and Wndchrm for color-normalized images.Conclusions:Color normalization can give a small incremental benefit when a super-pixel-based classification method is used with features that perform implicit color normalization while the gain is higher for patch-based classification methods for classifying epithelium versus stroma.

An alternative reference space for H&E color normalization

“…We introduced a scalar component to the classified image by measuring the color deviation of pixels about the mean for each tissue element. We found that this operation resulted in structure-specific intra-image variability not significantly different from the unnormalized image, and similar to that reported previously [17]. To test how well this residual variability reflects the original modulation of the image, we measured the normalized mutual information (NMI) between the normalized and unnormalized images in HSV space (Fig 5A, dark bars).…”

“…This analysis provided quantitative insight into the amount of color variability present in the original images and the variability that persisted after normalization, and has previously been used in other reports [17]. Despite the pursuit to significantly reduce unwanted inter-image variability, an ideal for most color normalization algorithms is that the inherent variability within images is preserved.…”

“…These algorithms shared a common strategy in that normalization can be approached from the standpoint of color analysis: that the superposition of stains that form the color basis of an image can be deconstructed into its constituents and recombined to conform to a target. A recent test [17] of two state-of-the-art algorithms demonstrated that color normalization can reduce the significant color variation across images by a factor of nearly 2, while retaining most of the intra-image pixel variance that may correspond to useful image details. Likewise, Bautista et al [9] recently showed that the dissimilarity in color that resulted from differences in whole slide scanners could be reduced by a factor of 3–4 using a color calibration technique.…”

“…Likewise, Bautista et al [9] recently showed that the dissimilarity in color that resulted from differences in whole slide scanners could be reduced by a factor of 3–4 using a color calibration technique. However, even with advanced normalization tools, residual variability across images persists; in the data set used by Sethi, et al [17], this residual inter-image variability after color normalization amounted to approximately 10% of the total range (estimated by the standard deviation that they reported). Given that the differences in the color properties of some important histological structures may often be even less than 10% [18, 19], improvements to reduce inter-image variability must therefore still be made in order to consistently and reliably distinguish between histological structures and interpret the tissue correctly.…”

A survey on deep learning in medical image analysis

“…Detection and classification of disease Cruz-Roa et al (2014) Detection of invasive ductal carcinoma H&E CNN-based patch classifier Xu et al (2014) Patch-level classification of colon cancer H&E Multiple instance learning framework with CNN features Bychkov et al (2016) Outcome prediction of colorectal cancer H&E Extracted CNN features from epithelial tissue for prediction Chang et al (2017) Multiple cancer tissue classification Various Transfer learning using multi-Scale convolutional sparse coding Günhan Ertosun and Rubin (2015) Grading glioma H&E Ensemble of CNNs Källén et al (2016) Predicting Gleason score H&E OverFeat pre-trained network as feature extractor Kim et al (2016a) Thyroid cytopathology classification H&E, RM & Pap Fine-tuning pre-trained AlexNet Litjens et al (2016) Detection of prostate and breast cancer H&E fCNN-based pixel classifier Quinn et al (2016) Malaria, tuberculosis and parasites detection Light microscopy CNN-based patch classifier Rezaeilouyeh et al (2016) Gleason grading and breast cancer detection H&E The system incorporates shearlet features inside a CNN Schaumberg et al (2016) SPOP mutation prediction of prostate cancer H&E Ensemble of ResNets Wang et al (2016b) Metastases Sethi et al (2016) Comparison of normalization algorithms H&E Presents effectiveness of stain normalization for application of CNNs CNNs with classical machine learning approaches using handcrafted features. In chest X-ray, several groups detect multiple diseases with a single system.…”