Synthesis of diagnostic quality cancer pathology images by generative adversarial networks

Abstract

Deep learning‐based computer vision methods have recently made remarkable breakthroughs in the analysis and classification of cancer pathology images. However, there has been relatively little investigation of the utility of deep neural networks to synthesize medical images. In this study, we evaluated the efficacy of generative adversarial networks to synthesize high‐resolution pathology images of 10 histological types of cancer, including five cancer types from The Cancer Genome Atlas and the five major histological subtypes of ovarian carcinoma. The quality of these images was assessed using a comprehensive survey of board‐certified pathologists (n = 9) and pathology trainees (n = 6). Our results show that the real and synthetic images are classified by histotype with comparable accuracies and the synthetic images are visually indistinguishable from real images. Furthermore, we trained deep convolutional neural networks to diagnose the different cancer types and determined that the synthetic images perform as well as additional real images when used to supplement a small training set. These findings have important applications in proficiency testing of medical practitioners and quality assurance in clinical laboratories. Furthermore, training of computer‐aided diagnostic systems can benefit from synthetic images where labeled datasets are limited (e.g. rare cancers). We have created a publicly available website where clinicians and researchers can attempt questions from the image survey (http://gan.aimlab.ca/). © 2020 Pathological Society of Great Britain and Ireland. Published by John Wiley & Sons, Ltd.

Generative models in pathology: synthesis of diagnostic quality pathology images†

“…In the clinical study, the pathologists answered questions about the histotype, the adequacy of image quality and the question of authenticity (real or synthetic). Levine, Peng et al [4] reported slightly better histotype classification performance for synthetic images over real samples in both datasets. In the pathologists' assessment, the fake images were sharper in both datasets, as they preferred synthetic images over real images in terms of quality for histologic evaluation.…”

“…Levine, Peng et al [4] highlight how generative models can be used in the pathology domain with existing computational resources and digitized datasets to produce large high‐quality images that are indistinguishable from real scans, even for experts. Furthermore, as the generator network has no direct access to the real images, Levine, Peng et al [4] suggest that the application of the generative models in combination with other privacy strategies can reduce the risks of patient privacy being compromised. Also, Levine, Peng et al [4] reported a few common GAN‐related artifacts that were encountered in the study and they presented some examples.…”

“…Furthermore, as the generator network has no direct access to the real images, Levine, Peng et al [4] suggest that the application of the generative models in combination with other privacy strategies can reduce the risks of patient privacy being compromised. Also, Levine, Peng et al [4] reported a few common GAN‐related artifacts that were encountered in the study and they presented some examples. However, the principal aspect of the paper is the clinical study that can raise interest among clinicians to employ existing machine learning and computer vision methods to improve healthcare.…”

“…To conclude, there are generally two ways generative modeling can be used for applications in medical imaging [9]. The first is the generative aspect, which has been explored in [4]. The proposed models have been shown to capture the underlying tissue structure of the training data and learning to generate realistic‐looking fake images.…”

“…For instance, by learning a distribution of normal anatomical variability using a GAN, the model can label and score an anomaly of new data according to its fit into the learned distribution [15]. Beyond all valuable insights that Levine, Peng et al [4] offer in their paper, due to the immensity of gigapixel images in digital pathology, future works are needed to improve the scalability of the existing generative methods to be applied on whole‐slide images [16], alongside studies to avoid or decrease unwanted artifacts in simulating synthetic pathology images.…”