Pan-cancer image-based detection of clinically actionable genetic alterations

Abstract

and tluedde@ukaachen.de 34 35 Precision treatment of cancer relies on genetic alterations which are diagnosed by molecular 36 biology assays. 1 These tests can be a bottleneck in oncology workflows because of high turna-37 round time, tissue usage and costs. 2 Here, we show that deep learning can predict point muta-38 tions, molecular tumor subtypes and immune-related gene expression signatures 3,4 directly 39 from routine histological images of tumor tissue. We developed and systematically optimized 40 a one-stop-shop workflow and applied it to more than 4000 patients with breast 5 , colon and 41 rectal 6 , head and neck 7 , lung 8,9 , pancreatic 10 , prostate 11 cancer, melanoma 12 and gastric 13 can-42 cer. Together, our findings show that a single deep learning algorithm can predict clinically ac-43 tionable alterations from routine histology data. Our method can be implemented on mobile 44 hardware 14 , potentially enabling point-of-care diagnostics for personalized cancer treatment 45 in individual patients. 46 Clinical guidelines recommend molecular testing of tumor tissue for most patients with advanced 47 209 The results are in part based upon data generated by the TCGA Research Network: http://can-210 cergenome.nih.gov/. Our funding sources are as follows. J.N.K.: RWTH University Aachen (START 211

Deep learning-based cross-classifications reveal conserved spatial behaviors within tumor histological images

“…While other recent works have investigated image-based cancer classification (Fu et al 2019;Kather et al 2019), cross-classification has until now been little studied. Comparisons of classifiers support the existence of morphological features shared across cancer types, as many cross-cancer predictors achieve high AUCs.…”

Histopathological Landscape of Molecular Genetics and Clinical Determinants in MDS Patients

“…Here, we demonstrate how the intricate and heterogeneous BM morphological landscape can be decomposed and associated with clinical data using multilevel computer vision. Remarkably, highest prediction accuracy of deep BM morphology was noted for mutation and cytogenetic aberrations, which even outweighed reported inference in solid tumors 5,9,17,18 . We suspect homogenous BM tissue consistency and lower mutation burden of MDS to account for the improved results.…”

Dense, high-resolution mapping of cells and tissues from pathology images for the interpretable prediction of molecular phenotypes in cancer

“…Recent work has revealed the weaknesses of low-interpretability models, including brittleness to dataset shift, vulnerabilities to adversarial attack, and susceptibility to the biases of the data-generative process. Unlike class activation maps utilized in prior studies as a heuristic to identify predictive image regions 9,10 , HIFs can be interpreted in aggregate across thousands of images and mapped directly onto biological concepts. Beyond suggesting interpretable hypotheses for causal mechanisms (e.g.…”

“…Over the past decade, the quantity and resolution of digitized histology slides has dramatically improved 6 . At the same time, the field of computer vision has made significant strides in pathology image analysis, including automated prediction of tumor grade 7 , mutational subtypes 8 , and gene expression signatures across cancer types 9,10 . In addition to achieving diagnostic sensitivity and specificity metrics that match or exceed those of human pathologists 11,12 , automated computational pathology can also scale to service resource-constrained settings where few pathologists are available.…”