Integrating morphologic and molecular histopathological features through whole slide image registration and deep learning

Abstract

Background Modern molecular pathology workflows in neuro-oncology heavily rely on the integration of morphologic and immunohistochemical patterns for analysis, classification, and prognostication. However, despite the recent emergence of digital pathology platforms and artificial intelligence-driven computational image analysis tools, automating the integration of histomorphologic information found across these multiple studies is challenged by large files sizes of whole slide images (WSIs) and shifts/rotations in tissue sections introduced during slide preparation. Methods To address this, we develop a workflow that couples different computer vision tools including scale-invariant feature transform (SIFT) and deep learning to efficiently align and integrate histopathological information found across multiple independent studies. We highlight the utility and automation potential of this workflow in the molecular subclassification and discovery of previously unappreciated spatial patterns in diffuse gliomas. Results First, we show how a SIFT-driven computer vision workflow was effective at automated WSI alignment in a cohort of 107 randomly selected surgical neuropathology cases (97/107 (91%) showing appropriate matches, AUC = 0.96). This alignment allows our AI-driven diagnostic workflow to not only differentiate different brain tumor types, but also integrate and carry out molecular subclassification of diffuse gliomas using relevant immunohistochemical biomarkers (IDH1-R132H, ATRX). To highlight the discovery potential of this workflow, we also examined spatial distributions of tumors showing heterogenous expression of the proliferation marker MIB1 and Olig2. This analysis helped uncovered an interesting and unappreciated association of Olig2 positive and proliferative areas in some gliomas (r = 0.62). Conclusion This efficient neuropathologist-inspired workflow provides a generalizable approach to help automate a variety of advanced immunohistochemically compatible diagnostic and discovery exercises in surgical neuropathology and neuro-oncology.

Insight into deep learning for glioma IDH medical image analysis: A systematic review

“…Deng et al (2023) [16] TCGA TCGA (n = 613) AUC (0.7737) CS-MIL No Loeffler et al (2022) [17] TCGA TCGA (n = 680) AUC (0.764) DenseNet No Wang et al (2023) [18] TCGA TCGA (n = 940) AUC (86.4) HMT-MIL (Customised architecture) No Faust et al (2022) [19] University of Toronto University of Toronto (n = 47) Accuracy (99.3%) VGG19 No Fang et al (2023) [20] TCGA Xiangya Hospital TCGA (n = 844) Xiangya (n = 116) AUC (0.827 ± 0.0465) Multi-Beholder (Customised architecture)…”

Standardizing analysis of intra‐tumoral heterogeneity with computational pathology

“…Over the past decade, there has also been an explosion of single cell and spatial profiling technologies that allow comprehensive evaluation of heterogeneity at the individual cell level. These include single cell profiling of dissociated cells, spatial transcriptomics/genomics, and imaging mass cytometry 70 . While these tools provide an unprecedented amount of molecular detail at high cellular resolution, they often are limited by superficial profiling depths (1000–2000 transcript IDs/cell) that usually only allow for assessment of cell‐type composition and enrichment analysis of major biological processes.…”

“…These include single cell profiling of dissociated cells, spatial transcriptomics/genomics, and imaging mass cytometry. 70 all of these technologies are also limited in the overall fraction of cells they can profile within a tumor, compared to bulk-tissue profiling techniques. Current approaches still only provide details on thousands of cells, a relatively small fraction for tumors that often contain billions of cells.…”

Glioma subtype classification from histopathological images using in-domain and out-of-domain transfer learning: An experimental study

“…Closely related to this task is the prediction of IDH1/2 mutation status, as an important diagnostic and prognostic biomarker in diffuse gliomas [13] . Other methods propose integrating features extracted from histopathological images with molecular features [14] , [15] , or combining WSIs with Magnetic Resonance Imaging (MRI) [16] , [17] .…”

“…These approaches dominantly use Convolutional Neural Networks (CNN) for extracting features from WSIs, either trained from scratch on the histopathological image dataset of interest [10] , [13] , [17] ; or using the transfer learning techniques with models pretrained in a domain of natural images [7] , [15] . Although CNNs are still considered to be state-of-the-art models for image classification, a change of paradigm can be recently observed towards the use of attention-based architectures and transformer networks in computer vision, challenging the superiority of CNNs.…”