Predicting cancer outcomes from histology and genomics using convolutional networks

Mobadersany, Pooya; Yousefi, Sahar; Amgad, Mohamed; Gutman, David; Barnholtz‐Sloan, Jill S.; Vega, José E. Velázquez; Brat, Daniel J.; Cooper, Lee

doi:10.1073/pnas.1717139115

Cited by 782 publications

(409 citation statements)

References 44 publications

Supporting

Mentioning

404

Contrasting

Unclassified

Order By: Relevance

“…Results by another group on the same dataset also confirmed better prognostication when Path images and genomic biomarkers (IDH, 1p/19q) were used together 15 . These data and some other studies on prostate cancer 16,17 support the hypothesis that combined evaluation of Rad and Path images will even further improve prognostication, and will enhance our understanding of the disease.…”

Section: Introductionmentioning

confidence: 60%

Nimg-76. Radiopathomics: Integration of Radiographic and Histologic Characteristics for Prognostication in Glioblastoma

et al. 2019

View full text Add to dashboard Cite

Both radiographic (Rad) imaging, such as multi-parametric magnetic resonance imaging, and digital pathology (Path) images captured from tissue samples are currently acquired as standard clinical practice for glioblastoma tumors. Both these data streams have been separately used for diagnosis and treatment planning, despite the fact that they provide complementary information. In this research work, we aimed to assess the potential of both Rad and Path images in combination and comparison. An extensive set of engineered features was extracted from delineated tumor regions in Rad images, comprising T1, T1-Gd, T2, T2-FLAIR, and 100 random patches extracted from Path images. Specifically, the features comprised descriptors of intensity, histogram, and texture, mainly quantified via gray-level-co-occurrence matrix and gray-level-run-length matrices. Features extracted from images of 107 glioblastoma patients, downloaded from The Cancer Imaging Archive, were run through support vector machine for classification using leaveone-out cross-validation mechanism, and through support vector regression for prediction of continuous survival outcome. The Pearson correlation coefficient was estimated to be 0.75, 0.74, and 0.78 for Rad, Path and RadPath data. The area-under the receiver operating characteristic curve was estimated to be 0.74, 0.76 and 0.80 for Rad, Path and RadPath data, when patients were discretized into long-and shortsurvival groups based on average survival cutoff. Our results support the notion that synergistically using Rad and Path images may lead to better prognosis at the initial presentation of the disease, thereby facilitating the targeted enrollment of patients into clinical trials.

show abstract

Section: Introductionmentioning

confidence: 60%

Nimg-76. Radiopathomics: Integration of Radiographic and Histologic Characteristics for Prognostication in Glioblastoma

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Changes in subcellular tissue structure can function as valuable biomarkers that can be used to assess onset and progression of disease. The use of digital pathology data in clinical and research settings have been studied and validated by several studies [7][8][9][10][11][12][13]. Availability of tissue images can facilitate multi-institutional and national level studies with large cohorts of patients.…”

Section: Image Analysis Tasks and Machine Learningmentioning

confidence: 99%

The Emergence of Pathomics

et al. 2019

View full text Add to dashboard Cite

Purpose of Review Our goal is to provide an overview of machine learning methods and artificial intelligence in digital pathology image analysis. We also highlight novel visualization tools to interpret quantitative image-based pathomics data that is extracted from whole slide images to describe diverse phenotypic characteristics of cancer in a spectrum of tissues. Recent Findings Image analysis of tissues is based on the identification and classification of tissue, architectural elements, cells, nuclei, and other histologic features. We report emerging digital pathology image analysis applications to study several types and subtypes of cancer to complement traditional histopathologic evaluation. Summary WSIs typically contain hundreds of thousands to millions of objects within a heterogeneous histologic landscape. Therefore, Pathomics represents an incredibly powerful emerging approach to classify cellular interactions and signaling by identifying relevant spatial relationships. The quantification of the intrinsic variability of different phenotypes and behavior in cancer is useful in analyzing and predicting clinical outcomes and treatment response.

show abstract

“…We replaced the last layer (i.e., linear classifier) in the model (as shown in Fig. 1) with a regression layer following the Cox proportional hazards model (Mobadersany et al, 2018;Fox, 2002). We used backpropagation to learn model parameters that maximize partial likelihood in the Cox model.…”

Section: Semi-supervised Clusteringmentioning

confidence: 99%

AffinityNet: Semi-Supervised Few-Shot Learning for Disease Type Prediction

Zhang

2019

AAAI

View full text Add to dashboard Cite

While deep learning has achieved great success in computer vision and many other fields, currently it does not work very well on patient genomic data with the "big p, small N " problem (i.e., a relatively small number of samples with high-dimensional features). In order to make deep learning work with a small amount of training data, we have to design new models that facilitate few-shot learning. Here we present the Affinity Network Model (AffinityNet), a data efficient deep learning model that can learn from a limited number of training examples and generalize well. The backbone of the AffinityNet model consists of stacked k-Nearest-Neighbor (kNN) attention pooling layers. The kNN attention pooling layer is a generalization of the Graph Attention Model (GAM), and can be applied to not only graphs but also any set of objects regardless of whether a graph is given or not. As a new deep learning module, kNN attention pooling layers can be plugged into any neural network model just like convolutional layers. As a simple special case of kNN attention pooling layer, feature attention layer can directly select important features that are useful for classification tasks. Experiments on both synthetic data and cancer genomic data from TCGA projects show that our AffinityNet model has better generalization power than conventional neural network models with little training data. We have implemented our method using PyTorch framework (https://pytorch.org).The code is freely available at https://github.com/BeautyOfWeb/AffinityNet.

show abstract

Predicting cancer outcomes from histology and genomics using convolutional networks

Cited by 782 publications

References 44 publications

Nimg-76. Radiopathomics: Integration of Radiographic and Histologic Characteristics for Prognostication in Glioblastoma

Nimg-76. Radiopathomics: Integration of Radiographic and Histologic Characteristics for Prognostication in Glioblastoma

The Emergence of Pathomics

AffinityNet: Semi-Supervised Few-Shot Learning for Disease Type Prediction

Contact Info

Product

Resources

About