Spatial Organization and Molecular Correlation of Tumor-Infiltrating Lymphocytes Using Deep Learning on Pathology Images

Saltz, Joel; Gupta, Rajarsi; Hou, Le; Kurç, Tahsin; Singh, Pankaj K.; Nguyen, Vu; Samaras, Dimitris; Shroyer, Kenneth R.; Zhao, Tianhao; Batiste, Rebecca; Arnam, John Van; Shmulevich, Ilya; Rao, Arvind; Lazar, Alexander J.; Sharma, Ashish; Thórsson, Vésteinn

doi:10.1016/j.celrep.2018.03.086

Cited by 776 publications

(726 citation statements)

References 56 publications

Supporting

Mentioning

664

Contrasting

Unclassified

Order By: Relevance

“…Pathological assessment of cancer specimens stained with hematoxylin and eosin (H&E) is primarily interpreted not only by tissue architecture but also by nuclear morphology of the tumor cells, which has been used for routine clinical diagnosis and computer‐aided pathological diagnosis . Recently, this field has significantly progressed to decipher clinical and biological relevance from such pathological images by combining molecular information such as genomic data …”

Section: Introductionmentioning

confidence: 99%

“…[11][12][13] Recently, this field has significantly progressed to decipher clinical and biological relevance from such pathological images by combining molecular information such as genomic data. 14,15 Gastric cancer is one of the most common human cancers, and is the second leading cause of cancer-related deaths worldwide. 16,17 As it is often associated with chronic inflammation caused by Helicobacter pylori infection and chemicals, 18 this disease is an example of human oncogenesis that is etiologically induced by environmental factors.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Computational analysis of morphological and molecular features in gastric cancer tissues

et al. 2020

View full text Add to dashboard Cite

Biological morphologies of cells and tissues represent their physiological and pathological conditions. The importance of quantitative assessment of morphological information has been highly recognized in clinical diagnosis and therapeutic strategies. In this study, we used a supervised machine learning algorithm wndchrm to classify hematoxylin and eosin (H&E)‐stained images of human gastric cancer tissues. This analysis distinguished between noncancer and cancer tissues with different histological grades. We then classified the H&E‐stained images by expression levels of cancer‐associated nuclear ATF7IP/MCAF1 and membranous PD‐L1 proteins using immunohistochemistry of serial sections. Interestingly, classes with low and high expressions of each protein exhibited significant morphological dissimilarity in H&E images. These results indicated that morphological features in cancer tissues are correlated with expression of specific cancer‐associated proteins, suggesting the usefulness of biomolecular‐based morphological classification.

show abstract

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

Computational analysis of morphological and molecular features in gastric cancer tissues

et al. 2020

View full text Add to dashboard Cite

show abstract

“…We then investigated how HE2RNA learned to recognize important histological patterns within the slides, by using our model to generate heatmaps corresponding to the most predictive tiles used to predict a gene's expression. Such approaches could also be used to detect histological subtypes 10 , genetic mutations 28 or to map the infiltration of tumors by tumor-infiltrating lymphocytes (TILs) 29 or other immune cells (e.g. macrophages, NK cells) based on cell-specific gene signatures .…”

mentioning

confidence: 99%

Transcriptomic learning for digital pathology

Schmauch¹,

Romagnoni²,

Pronier³

et al. 2019

Preprint

View full text Add to dashboard Cite

^,⋕ : These authors contributed equally.Deep learning methods for digital pathology analysis have proved an effective way to address multiple clinical questions, from diagnosis to prognosis and the prediction of treatment outcomes. They have also recently been used to predict gene mutations from pathology images, but no comprehensive evaluation of their potential for extracting molecular features from histology slides, has yet been performed. We propose a novel approach based on the integration of multiple data modes, and show that our deep learning model HE2RNA can be trained to predict systematically RNA-Seq profiles from whole-slide images alone, without the need for expert annotation. The model facilitates the virtual spatialization of gene expression, as validated by double-staining in an independent dataset. The results can therefore be interpreted in detail and this model opens up new opportunities for virtual staining. Finally, the transcriptomic representation learned by the model could be could be used to improve performances for other clinical tasks, particularly for small datasets. For example we studied the problem of predicting microsatellite instability from Hematoxylin & Eosin (H&E)-stained images. Greater prediction ability was achieved in such a realistic framework.

show abstract

“…In a recent study using TCGA samples across 13 different tumour types, but not testicular cancer, it was shown that deep learning–based TIL algorithms can identify TIL patterns that are linked to molecular features and outcome 17. In this study, an average area of 70 mm 2 of the tumour was analysed with the algorithm and in average more than 100 000 cells per sample were counted, which is several orders of magnitude more than would be feasible to count through visual assessment.…”

Section: Discussionmentioning

confidence: 99%

Deep learning for detecting tumour-infiltrating lymphocytes in testicular germ cell tumours

et al. 2018

View full text Add to dashboard Cite

AimsTo evaluate if a deep learning algorithm can be trained to identify tumour-infiltrating lymphocytes (TILs) in tissue samples of testicular germ cell tumours and to assess whether the TIL counts correlate with relapse status of the patient.MethodsTILs were manually annotated in 259 tumour regions from 28 whole-slide images (WSIs) of H&E-stained tissue samples. A deep learning algorithm was trained on half of the regions and tested on the other half. The algorithm was further applied to larger areas of tumour WSIs from 89 patients and correlated with clinicopathological data.ResultsA correlation coefficient of 0.89 was achieved when comparing the algorithm with the manual TIL count in the test set of images in which TILs were present (n=47). In the WSI regions from the 89 patient samples, the median TIL density was 1009/mm2. In seminomas, none of the relapsed patients belonged to the highest TIL density tertile (>2011/mm2). TIL quantifications performed visually by three pathologists on the same tumours were not significantly associated with outcome. The average interobserver agreement between the pathologists when assigning a patient into TIL tertiles was 0.32 (Kappa test) compared with 0.35 between the algorithm and the experts, respectively. A higher TIL density was associated with a lower clinical tumour stage, seminoma histology and lack of lymphovascular invasion.ConclusionsDeep learning–based image analysis can be used for detecting TILs in testicular germ cell cancer more objectively and it has potential for use as a prognostic marker for disease relapse.

show abstract

Spatial Organization and Molecular Correlation of Tumor-Infiltrating Lymphocytes Using Deep Learning on Pathology Images

Cited by 776 publications

References 56 publications

Computational analysis of morphological and molecular features in gastric cancer tissues

Computational analysis of morphological and molecular features in gastric cancer tissues

Transcriptomic learning for digital pathology

Deep learning for detecting tumour-infiltrating lymphocytes in testicular germ cell tumours

Contact Info

Product

Resources

About