Predicting survival from colorectal cancer histology slides using deep learning: A retrospective multicenter study

Abstract

BackgroundFor virtually every patient with colorectal cancer (CRC), hematoxylin–eosin (HE)–stained tissue slides are available. These images contain quantitative information, which is not routinely used to objectively extract prognostic biomarkers. In the present study, we investigated whether deep convolutional neural networks (CNNs) can extract prognosticators directly from these widely available images.Methods and findingsWe hand-delineated single-tissue regions in 86 CRC tissue slides, yielding more than 100,000 HE image patches, and used these to train a CNN by transfer learning, reaching a nine-class accuracy of >94% in an independent data set of 7,180 images from 25 CRC patients. With this tool, we performed automated tissue decomposition of representative multitissue HE images from 862 HE slides in 500 stage I–IV CRC patients in the The Cancer Genome Atlas (TCGA) cohort, a large international multicenter collection of CRC tissue. Based on the output neuron activations in the CNN, we calculated a “deep stroma score,” which was an independent prognostic factor for overall survival (OS) in a multivariable Cox proportional hazard model (hazard ratio [HR] with 95% confidence interval [CI]: 1.99 [1.27–3.12], p = 0.0028), while in the same cohort, manual quantification of stromal areas and a gene expression signature of cancer-associated fibroblasts (CAFs) were only prognostic in specific tumor stages. We validated these findings in an independent cohort of 409 stage I–IV CRC patients from the “Darmkrebs: Chancen der Verhütung durch Screening” (DACHS) study who were recruited between 2003 and 2007 in multiple institutions in Germany. Again, the score was an independent prognostic factor for OS (HR 1.63 [1.14–2.33], p = 0.008), CRC-specific OS (HR 2.29 [1.5–3.48], p = 0.0004), and relapse-free survival (RFS; HR 1.92 [1.34–2.76], p = 0.0004). A prospective validation is required before this biomarker can be implemented in clinical workflows.ConclusionsIn our retrospective study, we show that a CNN can assess the human tumor microenvironment and predict prognosis directly from histopathological images.

Accurate Diagnosis of Colorectal Cancer Based on Histopathology Images Using Artificial Intelligence

“…XH is the largest hospital in Hunan Province and was established in 1906 with a close affiliation with Yale University 25 . The other independent sources were TCGA of US (https://portal.gdc.cancer.gov/) 26 , NCT-UMM of Germany (https://zenodo.org/record/1214456#.XgaR00dTm00) 20 , Adicon Clinical Laboratories, INC (ACL), and eleven hospitals in China (detailed in Table 1). The hospitals involved are located in the major metropolitan areas of China serving >139 million population, including those most prestigious hospitals in pathology in China: XH, FUS, CGH, SWH, and AMU; other state-level esteemed hospitals: SYU, NJD, GPH, HPH, and TXH; and a regional reputable PCH.…”

Deep learning detects virus presence in cancer histology

“…All histological slides were reviewed and tumor regions were manually delineated in QuPath 19 , tessellated into tiles of 256 × 256 µm 2 which were subsequently downsampled to 224 × 224 px, yielding an effective magnification of 1.14 µm/px. These tumor tiles were used for deep transfer learning in MATLAB R2019a as described before 9,10 . We used a modified VGG19 deep convolutional neural network 20 which was pretrained on ImageNet (http://www.image-net.org, architecture shown in Suppl.…”

The coming 15 years in gynaecological pathology: digitisation, artificial intelligence, and new technologies

“…It is plausible that new classification or prognostication systems could emerge, not restricted to the traditional pathological features that we know so well. There has already been some early success in predicting survival from histology slides by using machine learning, potentially adding value to current TNM staging systems . Of course, it could turn out that what we are currently doing cannot be improved upon, but surely this cannot be the case in every scenario.…”