Deep Neural Network Analysis of Pathology Images With Integrated Molecular Data for Enhanced Glioma Classification and Grading

Abstract

Gliomas are primary brain tumors that originate from glial cells. Classification and grading of these tumors is critical to prognosis and treatment planning. The current criteria for glioma classification in central nervous system (CNS) was introduced by World Health Organization (WHO) in 2016. This criteria for glioma classification requires the integration of histology with genomics. In 2017, the Consortium to Inform Molecular and Practical Approaches to CNS Tumor Taxonomy (cIMPACT-NOW) was established to provide up-to-date recommendations for CNS tumor classification, which in turn the WHO is expected to adopt in its upcoming edition. In this work, we propose a novel glioma analytical method that, for the first time in the literature, integrates a cellularity feature derived from the digital analysis of brain histopathology images integrated with molecular features following the latest WHO criteria. We first propose a novel over-segmentation strategy for region-of-interest (ROI) selection in large histopathology whole slide images (WSIs). A Deep Neural Network (DNN)-based classification method then fuses molecular features with cellularity features to improve tumor classification performance. We evaluate the proposed method with 549 patient cases from The Cancer Genome Atlas (TCGA) dataset for evaluation. The cross validated classification accuracies are 93.81% for lower-grade glioma (LGG) and high-grade glioma (HGG) using a regular DNN, and 73.95% for LGG II and LGG III using a residual neural network (ResNet) DNN, respectively. Our experiments suggest that the type of deep learning has a significant impact on tumor subtype discrimination between LGG II vs. LGG III. These results outperform state-of-the-art methods in classifying LGG II vs. LGG III and offer competitive performance in distinguishing LGG vs. HGG in the literature. In addition, we also investigate molecular subtype classification using pathology images and cellularity information. Finally, for the first time in literature this work shows promise for cellularity quantification to predict brain tumor grading for LGGs with IDH mutations.

Deep Learning With Radiomics for Disease Diagnosis and Treatment: Challenges and Potential

“…In the relationship between tumor phenotypes and pathological characteristics, employing the radiomic signatures of ceCT and non-ceCT, WSIs, and pathology and CT can identify pathological biomarkers of pancreatic neuroendocrine tumors ( 112 ), diagnose and grade prostate cancer ( 71 ), and recognize NSCLC subtypes (apparent diffusion coefficient and squamous cell carcinoma) ( 124 ), respectively. The accuracy of the radiomic model based on depth characteristics of WSIs in predicting glioma grading (LGG and high-grade glioma) ( 147 ), liver cancer subtypes ( 148 ), and molecular subtypes of bladder cancer ( 90 ) has reached the level that can be assessed by pathologists. Multimodality spectral imaging (optical coherence tomography, malignant pleural mesothelioma, and LSRM) for morphology-molecular metabolism analytics has distinguished the pituitary from tumor and classified pituitary adenoma subtypes ( 125 ).…”

“…Evidence supports that the classification and grading of many tumors, such as breast, colorectal, prostate, glioma, and lung cancer, are possible through histopathological images (21,71,88,146,147). Specifically, Sharma and Mehra (146) evaluated the discriminative power of handcrafted and baseline pathology and depth features in a breast cancer multi-classification problem, with linear SVM and VGG16 networks exhibiting excellent predictive performance.…”

“…Pei et al. ( 147 ) developed a deep neural network model incorporating molecular and cellular characteristics to differentiate LGG and high-grade glioma. This algorithm reportedly outperforms state-of-the-art methods in detecting LGG II and LGG III and performs better in distinguishing LGG from high-grade glioma.…”

Combining Radiology and Pathology for Automatic Glioma Classification

“…Then, the OTSU method is adopted to remove non-tissue regions ( Otsu, 1979 ). Following the current studies ( Ma and Jia, 2019 ; Pei et al, 2019 ; Pei et al, 2020 ; Pei et al, 2021 ), we exclude meaningless tissues using a simple but effective threshold technique. Specifically, we first calculate the mean value and standard deviation of each patch in RGB space and maintain patches with a mean value between 100 and 220 and standard deviations above 20 ( Pei et al, 2019 ; Pei et al, 2020 ; Pei et al, 2021 ).…”

“…Following the current studies ( Ma and Jia, 2019 ; Pei et al, 2019 ; Pei et al, 2020 ; Pei et al, 2021 ), we exclude meaningless tissues using a simple but effective threshold technique. Specifically, we first calculate the mean value and standard deviation of each patch in RGB space and maintain patches with a mean value between 100 and 220 and standard deviations above 20 ( Pei et al, 2019 ; Pei et al, 2020 ; Pei et al, 2021 ). Then, we convert each patch to the hue saturation value (HSV) space and exclude patches with the mean value below 50 in the H channel ( Ma and Jia, 2019 ).…”