A novel approach combined transfer learning and deep learning to predict TMB from histology image

Wang, Liansheng; Jiao, Yudi; Qiao, Ying; Zeng, Nianyin; Yu, Rongshan

doi:10.1016/j.patrec.2020.04.008

Cited by 32 publications

(21 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In another work, Wang and colleagues also attempted to classify TMB status from FFPE slides for the gastrointestinal cohorts from the TCGA (n = 545; ref. 45). Like Jain and colleagues, this group also relied on TMB calculated from nonsynonymous mutation counts from whole exomes and used the upper tertile as the cutoff to define high TMB.…”

Section: Making the Most Of Mutationsmentioning

confidence: 99%

Artificial Intelligence in Cancer Research and Precision Medicine

Bhinder

Gilvary²,

Madhukar³

et al. 2021

Cancer Discovery

363

153

View full text Add to dashboard Cite

Convolutional neural networks (CNN) have been the most popular deep learning architectures used for image classification in cancer ( Fig. 1 ). CNNs apply a series of nonlinear transformations to structured data (such as raw pixels of an image) to learn relevant features automatically, unlike conventional machine learning models that frequently require manual feature curation. On the fl ip side, it is diffi cult to tell what features are learned by the CNNs, making them what many have referred to as a "black box." One consequence is that images used for CNNs should be carefully prepro cessed to reduce the risk that the model learns from image artifacts. There are two major approaches for CNN models; one is transfer learning that uses images from a large collection of natural objects (such as in ImageNet) to train the initial layers of a model (where the model learns to identify general features such as shapes and edges) and then uses the diseasespecifi c data to fi ne-tune the training parameters in the last layers. The second variation of CNN is based on an autoencoder where the model learns background features from a subset of representative images and encodes a compressed representation of the basic features later used to initialize the CNN. In the CAMELYON16 Challenge-a crowdsourced competition to identify and classify lymph node metastasis in patients with breast cancer from whole slide images (WSI) of hematoxylin and eosin (H&E)-stained tumors-25 of the 32 submitted algorithms were CNNs, and the top 5 classifi cation models were exclusively based on transfer learning, which were GoogLeNet, ResNet, VGG-16 ( 2 ). Khosravi and colleagues trained and tested several state-of-the-art deep learning models to classify WSI from H&E-stained tumor tissues of The Cancer Genome Atlas (TCGA) cohort and reported on the

show abstract

Section: Making the Most Of Mutationsmentioning

confidence: 99%

Artificial Intelligence in Cancer Research and Precision Medicine

Bhinder

Gilvary²,

Madhukar³

et al. 2021

Cancer Discovery

363

153

View full text Add to dashboard Cite

show abstract

“…In this study, we explored the ability to rapidly estimate TMB status in lung adenocarcinoma using H&E histopathology images combined with clinico-demographic data. The resulting end to end system provides both histologic subtype classification and interpretable TMB status estimation with an AUC of 0.77 on a set of slides from held out sites, significantly better than using the baseline clinical features alone (AUC of 0.66 for the same cases) and on par with TMB prediction models recently reported for other cancer types from TCGA 9 , 25 .…”

Section: Discussionmentioning

confidence: 89%

“…Several recent efforts have explored the potential for deep learning algorithms to perform a range of tasks in pathology, including the detection of tumor, the classification of tumor histology, and more recently, the prediction of molecular biomarker status 6 – 9 . These approaches show promise, but the histopathological features being learned and used by the algorithms are often unknown and cannot be quality-controlled or readily verified by researchers and pathologists.…”

Section: Introductionmentioning

confidence: 99%

Comparative analysis of machine learning approaches to classify tumor mutation burden in lung adenocarcinoma using histopathology images

Sadhwani

Chang

Behrooz

et al. 2021

Sci Rep

View full text Add to dashboard Cite

Both histologic subtypes and tumor mutation burden (TMB) represent important biomarkers in lung cancer, with implications for patient prognosis and treatment decisions. Typically, TMB is evaluated by comprehensive genomic profiling but this requires use of finite tissue specimens and costly, time-consuming laboratory processes. Histologic subtype classification represents an established component of lung adenocarcinoma histopathology, but can be challenging and is associated with substantial inter-pathologist variability. Here we developed a deep learning system to both classify histologic patterns in lung adenocarcinoma and predict TMB status using de-identified Hematoxylin and Eosin (H&E) stained whole slide images. We first trained a convolutional neural network to map histologic features across whole slide images of lung cancer resection specimens. On evaluation using an external data source, this model achieved patch-level area under the receiver operating characteristic curve (AUC) of 0.78–0.98 across nine histologic features. We then integrated the output of this model with clinico-demographic data to develop an interpretable model for TMB classification. The resulting end-to-end system was evaluated on 172 held out cases from TCGA, achieving an AUC of 0.71 (95% CI 0.63–0.80). The benefit of using histologic features in predicting TMB is highlighted by the significant improvement this approach offers over using the clinical features alone (AUC of 0.63 [95% CI 0.53–0.72], p = 0.002). Furthermore, we found that our histologic subtype-based approach achieved performance similar to that of a weakly supervised approach (AUC of 0.72 [95% CI 0.64–0.80]). Together these results underscore that incorporating histologic patterns in biomarker prediction for lung cancer provides informative signals, and that interpretable approaches utilizing these patterns perform comparably with less interpretable, weakly supervised approaches.

show abstract

“…Instead, cohorts are typically stratified into TMB-low and TMB-high groups based on a defined threshold, thus resulting in a binary classification problem which can be tackled using a neural network. The first published study found to apply these methods was carried out by Wang et al in the context of gastric and colon cancer [64]. The performance of eight state-of the-art deep learning models was compared on TCGA stomach adenocarcinoma (STAD) and colon adenocarcinoma (COAD) data sets separately.…”

Section: Predicting Tumour Mutational Burdenmentioning

confidence: 99%

Deep Learning of Histopathological Features for the Prediction of Tumour Molecular Genetics

et al. 2021

View full text Add to dashboard Cite

Advanced diagnostics are enabling cancer treatments to become increasingly tailored to the individual through developments in immunotherapies and targeted therapies. However, long turnaround times and high costs of molecular testing hinder the widespread implementation of targeted cancer treatments. Meanwhile, gold-standard histopathological assessment carried out by a trained pathologist is widely regarded as routine and mandatory in most cancers. Recently, methods have been developed to mine hidden information from histopathological slides using deep learning applied to scanned and digitized slides; deep learning comprises a collection of computational methods which learn patterns in data in order to make predictions. Such methods have been reported to be successful in a variety of cancers for predicting the presence of biomarkers such as driver mutations, tumour mutational burden, and microsatellite instability. This information could prove valuable to pathologists and oncologists in clinical decision making for cancer treatment and triage for in-depth sequencing. In addition to identifying molecular features, deep learning has been applied to predict prognosis and treatment response in certain cancers. Despite reported successes, many challenges remain before the clinical implementation of such diagnostic strategies in the clinical setting is possible. This review aims to outline recent developments in the field of deep learning for predicting molecular genetics from histopathological slides, as well as to highlight limitations and pitfalls of working with histopathology slides in deep learning.

show abstract

A novel approach combined transfer learning and deep learning to predict TMB from histology image

Cited by 32 publications

References 11 publications

Artificial Intelligence in Cancer Research and Precision Medicine

Artificial Intelligence in Cancer Research and Precision Medicine

Comparative analysis of machine learning approaches to classify tumor mutation burden in lung adenocarcinoma using histopathology images

Deep Learning of Histopathological Features for the Prediction of Tumour Molecular Genetics

Contact Info

Product

Resources

About