Normalization of HE-stained histological images using cycle consistent generative adversarial networks

Abstract

Background Histological images show strong variance (e.g. illumination, color, staining quality) due to differences in image acquisition, tissue processing, staining, etc. This can impede downstream image analysis such as staining intensity evaluation or classification. Methods to reduce these variances are called image normalization techniques. Methods In this paper, we investigate the potential of CycleGAN (cycle consistent Generative Adversarial Network) for color normalization in hematoxylin-eosin stained histological images using daily clinical data with consideration of the variability of internal staining protocol variations. The network consists of a generator network GB that learns to map an image X from a source domain A to a target domain B, i.e. GB:XA→XB. In addition, a discriminator network DB is trained to distinguish whether an image from domain B is real or generated. The same process is applied to another generator-discriminator pair (GA,DA), for the inverse mapping GA:XB→XA. Cycle consistency ensures that a generated image is close to its original when being mapped backwards (GA(GB(XA))≈XA and vice versa). We validate the CycleGAN approach on a breast cancer challenge and a follicular thyroid carcinoma data set for various stain variations. We evaluate the quality of the generated images compared to the original images using similarity measures. In addition, we apply stain normalization on pathological lymph node data from our institute and test the gain from normalization on a ResNet classifier pre-trained on the Camelyon16 data set. Results Qualitative results of the images generated by our network are compared to original color distributions. Our evaluation indicates that by mapping images to a target domain, the similarity training images from that domain improves up to 96%. We also achieve a high cycle consistency for the generator networks by obtaining similarity indices greater than 0.9. When applying the CycleGAN normalization to HE-stain images from our institute the kappa-value of the ResNet-model that is only trained on Camelyon16 data is increased more than 50%. Conclusions CycleGANs have proven to efficiently normalize HE-stained images. The approach compensates for deviations resulting from image acquisition (e.g. different scanning devices) as well as from tissue staining (e.g. different staining protocols), and thus overcomes the staining variations in images from various institutions.The code is publicly available at https://github.com/m4ln/stainTransfer_CycleGAN_pytorch. The data set supporting the solutions is available at 10.11588/data/8LKEZF.

Generative Adversarial Networks Accurately Reconstruct Pan-Cancer Histology from Pathologic, Genomic, and Radiographic Latent Features

“…Furthermore, we demonstrate that models that are highly confounded by site-specific factors such as ancestry can be quickly identified with HistoXGAN 14,37 . Interestingly, standard stain normalization such as the Reinhard 38 method demonstrate ‘overcorrection’ of color, as HistoXGAN illustrates models trained after Reinhard normalization identify the inverse color transition as associated with site, whereas CycleGAN normalization 39 was much more effective at eliminating learned staining patterns of tissue submitting sites.…”

Densely Convolutional Spatial Attention Network for nuclei segmentation of histological images for computational pathology

“…In tissue engineering, stain normalization is an important part of pre-processing before the analysis is performed. Because of differences in image acquisition, tissue processing, staining protocols, and the response function of digital scanners, histopathological images vary greatly (e.g., in illumination, color, and quality of stain) ( 48 ). This variation is a major issue for CNN-based computational pathology methods.…”

Computational Pathology for Brain Disorders

“…The framework consists of a GAN 2 (teacher network) trained to learn the mapping relationship between a source and target image; and an FCNN 3 (student network) able to transfer the mapping relationship of the GAN based on image content into a mapping relationship based on pixel values. A similar approach using cycle-consistent GANs was also proposed for the normalization of H&E-stained WSIs [38]. In the last case, synthetically generated images capture the representative variability in the color space of the WSI, enabling the architecture to transfer any color information from a new source image into a target color space.…”