Deep learning generates synthetic cancer histology for explainability and education

Abstract

Artificial intelligence methods including deep neural networks (DNN) can provide rapid molecular classification of tumors from routine histology with accuracy that matches or exceeds human pathologists. Discerning how neural networks make their predictions remains a significant challenge, but explainability tools help provide insights into what models have learned when corresponding histologic features are poorly defined. Here, we present a method for improving explainability of DNN models using synthetic histology generated by a conditional generative adversarial network (cGAN). We show that cGANs generate high-quality synthetic histology images that can be leveraged for explaining DNN models trained to classify molecularly-subtyped tumors, exposing histologic features associated with molecular state. Fine-tuning synthetic histology through class and layer blending illustrates nuanced morphologic differences between tumor subtypes. Finally, we demonstrate the use of synthetic histology for augmenting pathologist-in-training education, showing that these intuitive visualizations can reinforce and improve understanding of histologic manifestations of tumor biology.

Generative Adversarial Networks Accurately Reconstruct Pan-Cancer Histology from Pathologic, Genomic, and Radiographic Latent Features

“…In recent years, deep learning has been applied to predict molecular features of cancers directly from histology with varying degrees of accuracy 6,9,26 – but understanding the basis of these predictions remains challenging. Conditional GANs have been used to understand clear-cut histologic features, but must be retrained for each class comparison and cannot demonstrate multiple transitions simultaneously 27 . As HistoXGAN accurately recapitulates histologic features that are easily interpreted by pathologists like cancer grade and subtype, it can likely be applied for discovery of histologic patterns associated with molecular pathways.…”

“…4 As the field has evolved, studies have moved beyond basic pattern recognition towards identifying deeper disease traits and complex morphological features, including the identification of genomic and transcriptomic profiles directly from histology. 5–7 Conceptually, deep learning models often condense complex visual information from histopathology into a small number of higher order features for prediction; often using pretraining from large image datasets like ImageNet 8 or feature extractors trained with self-supervised learning (SSL) 9,10 . However, the opacity of these high level features limits adoption and deployment due to concerns about model trustworthiness 11 , and lack of interpretability limits the ability to gain new insight from the histologic patterns recognized by models.…”

“…12 Conditional GANs can be used to interpolate between two histologic classes – but training is time consuming and such an approach can only embed a limited number of classes. 5…”

Identification of age-dependent features of human bronchi using explainable artificial intelligence

“…Further studies are needed to definitely identify which parameters of bronchial bifurcation sites are quantitatively captured. For example, synthetic histology generated by a conditional generative adversarial network identified histological parameters associated with molecular state of tumours [ 26 ]. Such algorithms can be applied to bronchoscopy images.…”

Application of Example-Based Explainable Artificial Intelligence (XAI) for Analysis and Interpretation of Medical Imaging: A Systematic Review

“…Two recent studies on Generative Adversarial Networks (GANs) to create explanatory examples stood out for their significant advances in digital pathology and medicine. In the work of James M. Dolezal et al [12], a conditional Generative Adversarial Network (cGAN) based on the StyleGAN2 architecture was specialised to produce detailed histological images. The structure consists of a generator, responsible for creating realistic images, and a discriminator, which evaluates the authenticity of these images.…”