Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer

Kather, Jakob Nikolas; Pearson, Alexander T.; Halama, Niels; Jäger, Dirk; Krause, Jeremias; Loosen, Sven H.; Marx, Alexander; Boor, Peter; Tacke, Frank; Neumann, Ulf; Grabsch, Heike; Yoshikawa, Takaki; Brenner, Hermann; Chang‐Claude, Jenny; Hoffmeister, Michael; Trautwein, Christian; Luedde, Tom

doi:10.1038/s41591-019-0462-y

Cited by 987 publications

(922 citation statements)

References 25 publications

Supporting

Mentioning

886

Contrasting

Unclassified

Order By: Relevance

“…There is, therefore, a great need to screen standard WSIs from patients with solid tumors with a high probability of MSI-H directly, to facilitate access to immunotherapy. A recent study 54 showed that CNNs can learn to predict MSI status directly from histology slides for stomach adenocarcinoma and colorectal cancer. Based on these results, we collected, from the TCGA-COAD dataset, RNA-Seq measurements, WSIs and the corresponding MSI status of each patient from the dataset, to investigate the effects of integrating RNA-Seq information on the prediction of MSI status from pathology images.…”

Section: A Deep Learning Model For the Prediction Of Gene Expressionmentioning

confidence: 99%

Transcriptomic learning for digital pathology

Schmauch¹,

Romagnoni²,

Pronier³

et al. 2019

Preprint

View full text Add to dashboard Cite

^,⋕ : These authors contributed equally.Deep learning methods for digital pathology analysis have proved an effective way to address multiple clinical questions, from diagnosis to prognosis and the prediction of treatment outcomes. They have also recently been used to predict gene mutations from pathology images, but no comprehensive evaluation of their potential for extracting molecular features from histology slides, has yet been performed. We propose a novel approach based on the integration of multiple data modes, and show that our deep learning model HE2RNA can be trained to predict systematically RNA-Seq profiles from whole-slide images alone, without the need for expert annotation. The model facilitates the virtual spatialization of gene expression, as validated by double-staining in an independent dataset. The results can therefore be interpreted in detail and this model opens up new opportunities for virtual staining. Finally, the transcriptomic representation learned by the model could be could be used to improve performances for other clinical tasks, particularly for small datasets. For example we studied the problem of predicting microsatellite instability from Hematoxylin & Eosin (H&E)-stained images. Greater prediction ability was achieved in such a realistic framework.

show abstract

Section: A Deep Learning Model For the Prediction Of Gene Expressionmentioning

confidence: 99%

Transcriptomic learning for digital pathology

Schmauch¹,

Romagnoni²,

Pronier³

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…As a tool for feature extraction from images, we used deep learning, a form of artificial intelligence (AI), which has previously been used to detect high-level morphological features directly from histological images. 8-10…”

Section: Introductionmentioning

confidence: 99%

Deep learning detects virus presence in cancer histology

Kather

Schulte

Grabsch

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

1Oncogenic viruses like human papilloma virus (HPV) or Epstein Barr virus (EBV) are a major cause of human 2 cancer. Viral oncogenesis has a direct impact on treatment decisions because virus-associated tumors can 3 demand a lower intensity of chemotherapy and radiation or can be more susceptible to immune check-4 point inhibition. However, molecular tests for HPV and EBV are not ubiquitously available. 5We hypothesized that the histopathological features of virus-driven and non-virus driven cancers are suf-6 ficiently different to be detectable by artificial intelligence (AI) through deep learning-based analysis of 7 images from routine hematoxylin and eosin (HE) stained slides. We show that deep transfer learning can 8 predict presence of HPV in head and neck cancer with a patient-level 3-fold cross validated area-under-9 the-curve (AUC) of 0.89 [0.82; 0.94]. The same workflow was used for Epstein-Barr virus (EBV) driven 10 gastric cancer achieving a cross-validated AUC of 0.80 [0.70; 0.92] and a similar performance in external 11 validation sets. Reverse-engineering our deep neural networks, we show that the key morphological fea-12 tures can be made understandable to humans. 13 This workflow could enable a fast and low-cost method to identify virus-induced cancer in clinical trials or 14 clinical routine. At the same time, our approach for feature visualization allows pathologists to look into 15 the black box of deep learning, enabling them to check the plausibility of computer-based image classifi-16 cation. 17 18

show abstract

“…In domains like computer vision or natural language processing, deep neural networks have already dramatically improved state-of-the-art prediction performances (Goodfellow et al 2016;LeCun et al 2015). However, in the application of neuroimaging data analysis, a similar revolution has not materialized for most common prediction goals, despite considerable research effort and few successful exceptions, such as for the goal of image segmentation (Choi & Jin 2016;Kamnitsas et al 2017;Li et al 2018;Kather et al 2019) and image registration (Balakrishnan et al 2018;Yang et al 2017).…”

Section: Deep Learning Did Not Universally Improve Prediction Performmentioning

confidence: 99%

Deep learning for brains?: Different linear and nonlinear scaling in UK Biobank brain images vs. machine-learning datasets

Schulz

Yeo

Vogelstein

et al. 2019

Preprint

View full text Add to dashboard Cite

In recent years, deep learning has unlocked unprecedented success in various domains, especially in image, text, and speech processing. These breakthroughs may hold promise for neuroscience and especially for brain-imaging investigators who start to analyze thousands of participants. However, deep learning is only beneficial if the data have nonlinear relationships and if they are exploitable at currently available sample sizes. We systematically profiled the performance of deep models, kernel models, and linear models as a function of sample size on UK Biobank brain images against established machine learning references. On MNIST and Zalando Fashion, prediction accuracy consistently improved when escalating from linear models to shallow-nonlinear models, and further improved when switching to deep-nonlinear models. The more observations were available for model training, the greater the performance gain we saw. In contrast, using structural or functional brain scans, simple linear models performed on par with more complex, highly parameterized models in age/sex prediction across increasing sample sizes. In fact, linear models kept improving as the sample size approached ~10,000 participants.Our results indicate that the increase in performance of linear models with additional data does not saturate at the limit of current feasibility. Yet, nonlinearities of common brain scans remain largely inaccessible to both kernel and deep learning methods at any examined scale. Exploratory Research Space (OPSF449), RWTH Aachen. Simulations were performed with computing resources granted by RWTH Aachen University under project number "rwth0238".

show abstract

Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer

Abstract: This is a repository copy of Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer.

Cited by 987 publications

References 25 publications

Transcriptomic learning for digital pathology

Transcriptomic learning for digital pathology

Deep learning detects virus presence in cancer histology

Deep learning for brains?: Different linear and nonlinear scaling in UK Biobank brain images vs. machine-learning datasets

Contact Info

Product

Resources

About