A High-Performance System for Robust Stain Normalization of Whole-Slide Images in Histopathology

Abstract

Stain normalization is an important processing task for computer-aided diagnosis (CAD) systems in modern digital pathology. This task reduces the color and intensity variations present in stained images from different laboratories. Consequently, stain normalization typically increases the prediction accuracy of CAD systems. However, there are computational challenges that this normalization step must overcome, especially for real-time applications: the memory and run-time bottlenecks associated with the processing of images in high resolution, e.g., 40X. Moreover, stain normalization can be sensitive to the quality of the input images, e.g., when they contain stain spots or dirt. In this case, the algorithm may fail to accurately estimate the stain vectors. We present a high-performance system for stain normalization using a state-of-the-art unsupervised method based on stain-vector estimation. Using a highly-optimized normalization engine, our architecture enables high-speed and large-scale processing of high-resolution whole-slide images. This optimized engine integrates an automated thresholding technique to determine the useful pixels and uses a novel pixel-sampling method that significantly reduces the processing time of the normalization algorithm. We demonstrate the performance of our architecture using measurements from images of different sizes and scanner formats that belong to four different datasets. The results show that our optimizations achieve up to 58x speedup compared to a baseline implementation. We also prove the scalability of our system by showing that the processing time scales almost linearly with the amount of tissue pixels present in the image. Furthermore, we show that the output of the normalization algorithm can be adversely affected when the input images include artifacts. To address this issue, we enhance the stain normalization pipeline by introducing a parameter cross-checking technique that automatically detects the distortion of the algorithm's critical parameters. To assess the robustness of the proposed method we employ a machine learning (ML) pipeline that classifies images for detection of prostate cancer. The results show that the enhanced normalization algorithm increases the classification accuracy of the ML pipeline in the presence of poor-quality input images. For an exemplary ML pipeline, our new method increases the accuracy on an unseen dataset from 0.79 to 0.87.

Artificial Intelligence-Based Multiclass Classification of Benign or Malignant Mucosal Lesions of the Stomach

“…To overcome the staining inconsistency of histology slides, multiple researchers have applied operations to standardize specimen colors in histopathological images prior to analysis ( Ranefall et al, 1997 ; Reinhard et al, 2001 ; Macenko et al, 2009 ; Tam et al, 2016 ; Vahadane et al, 2016 ; Anghel et al, 2019 ; Ren et al, 2019 ). One common approach to tackle stain normalization issue is to extract multiple affinities for specific biological substances, and then perform some kind of projection from a preselected reference image to all images.…”

“…One common approach to tackle stain normalization issue is to extract multiple affinities for specific biological substances, and then perform some kind of projection from a preselected reference image to all images. Specifically, color deconvolution methods ( Macenko et al, 2009 ; Vahadane et al, 2016 ; Anghel et al, 2019 ) have been utilized extensively in the past decades by transforming the original RGB image into other color space like Lab ( Reinhard et al, 2001 ) and extract the stain vectors. Unsupervised vector estimation methods ( Anghel et al, 2019 ) and generative methods ( Ren et al, 2019 ) have also emerged in the past years.…”

The Impact of Digital Histopathology Batch Effect on Deep Learning Model Accuracy and Bias

“…Color augmentation (Figure 1), where the color channels of images are altered at random during training to prevent a model from learning stain characteristics of a specific site have also been utilized in histology deep learning tasks 28,29 . Most assessments of stain normalization and augmentation techniques have focused on the performance of models in validation sets, rather than true elimination of batch effect 30,31 . Here, we describe the clinical and slide level variability between sites in TCGA, and methods to ensure robust use of internal and external validation to minimize false positive findings with deep learning image analysis.…”

“…Finally, stain normalization and color augmentation techniques should still be used to improve model accuracy in external validation and implementation. Although normalization and augmentation do not completely prevent models from learning site specific characteristics, several studies have reported greater validation accuracies with the use of such techniques 30,31 . It is likely that these techniques eliminate some but not all of the reliance that deep learning models have on stain related features; by making the differences in slide characteristics more subtle, models may be more likely to pick up on biologically relevant factors.…”

“…Scanned whole slide images of hematoxylin and eosin stained tissue were acquired in SVS format. Each slide was reviewed by a pathologist for manual annotation of area of tumor, to ensure ink or other noncancer artifacts did not influence slide level statistics 31 . For slide level first order and second order statistical analysis, slides were downsampled to 5 microns per pixel, or approximately 2X magnification.…”