Deep learning detects genetic alterations in cancer histology generated by adversarial networks

Abstract

Deep learning can detect microsatellite instability (MSI) from routine histology images in colorectal cancer (CRC). However, ethical and legal barriers impede sharing of images and genetic data, hampering development of new algorithms for detection of MSI and other biomarkers. We hypothesized that histology images synthesized by conditional generative adversarial networks (CGANs) retain information about genetic alterations. To test this, we developed a ‘histology CGAN’ which was trained on 256 patients (training cohort 1) and 1457 patients (training cohort 2). The CGAN synthesized 10 000 synthetic MSI and non‐MSI images which contained a range of tissue types and were deemed realistic by trained observers in a blinded study. Subsequently, we trained a deep learning detector of MSI on real or synthetic images and evaluated the performance of MSI detection in a held‐out set of 142 patients. When trained on real images from training cohort 1, this system achieved an area under the receiver operating curve (AUROC) of 0.742 [0.681, 0.854]. Training on the larger cohort 2 only marginally improved the AUROC to 0.757 [0.707, 0.869]. Training on purely synthetic data resulted in an AUROC of 0.743 [0.658, 0.801]. Training on both real and synthetic data further increased AUROC to 0.777 [0.715, 0.821]. We conclude that synthetic histology images retain information reflecting underlying genetic alterations in colorectal cancer. Using synthetic instead of real images to train deep learning systems yields non‐inferior classifiers. This approach can be used to create large shareable data sets or to augment small data sets with rare molecular features. © 2021 The Authors. The Journal of Pathology published by John Wiley & Sons, Ltd. on behalf of The Pathological Society of Great Britain and Ireland.

Deep Learning of Histopathological Features for the Prediction of Tumour Molecular Genetics

“…Therefore, future studies will likely rely on collaborations between multiple research centres to attain the level of performance and testing required to eventually enable safe and effective clinical decisions to be made based on molecular biomarkers detected through deep learning applied to histopathology slides. Additionally, Krause et al and Liu et al's use of synthetic images to augment digital pathology data sets has been shown to merit further research and such methods may prove invaluable in the future when assembling large, shareable data sets [27,41].…”

“…Deep learning models in digital pathology return predictions at the tile-level; therefore, it is necessary to implement a strategy of aggregating per-tile predictions in order to obtain a per-slide prediction. Methods of tile aggregation range from majority voting to MIL approaches [27,28].…”

“…Another recent study sought to investigate how synthetic images could be used to augment data sets for training deep learning models to predict MSI status in colorectal cancer [27]. The authors were able to show that images produced using a GAN retained histomorphological features associated with MSI, such as poor differentiation and intraepithelial lymphocytosis.…”

Adversarial Learning of Cancer Tissue Representations

“…Within unsupervised models, Generative Adversarial Networks (GANs) have been widely used in digital pathology, from nuclei segmentation [14], stain transformation and normalization [18,23], to high-quality tissue samples [12]. In addition, there has been some initial work on building representations of cells [5] or larger tissue patches [16].…”

Deep Convolutional Neural Networks Detect Tumor Genotype from Pathological Tissue Images in Gastrointestinal Stromal Tumors

“…One approach to reducing the time required to diagnose drug-sensitive mutation is the use of deep learning algorithms to provide a suggestion of mutations directly from tumor features. Several studies have addressed this issue of morphology–genotype correlation in cancers [ 1 , 2 , 3 , 4 , 5 , 6 ]. However, in these studies, researchers primarily focused on identifying the presence of specific mutations within carcinomas, without taking into account the knowledge that, in some circumstances, the mutational subtype, not merely the presence or absence of a mutation, is the main determinant of effective treatment.…”