Abstract 1218: HOPE2Net: Integrating histological features and position embeddings in spatially resolved transcriptomics to predict gene expression and pathway activities from histology images in tumors

Abstract

Recent developments in spatially resolved transcriptomics (SRT) technologies have enabled the profiling of transcriptome-wide gene expression while retaining spatial location information of each measured spot within a tissue. Meanwhile, the corresponding histopathology images of tissue sections are readily available and can be aligned to the measured spots. Given that the histology images are practically more convenient and affordable to obtain, we designed HOPE2Net, a multi-layer perceptron architecture, that leverages information provided by SRT data to predict gene expression and pathway activities from histology images. Through systematic evaluations of different approaches for extracting deep image features and cellular morphology features, HOPE2Net performs feature selections from a pre-trained Vision Transformer, which is the state-of-art deep learning model for image recognition. After extracting histological image features, HOPE2Net further integrates with position embeddings, to optimize the gene expression and pathway activity prediction tasks. Through analyzing breast cancer and prostate cancer SRT datasets obtained from numerous tissue sections in multiple patients, we demonstrate that HOPE2Net can accurately predict the gene expression patterns for highly variable genes and the activities for significantly enriched domain-specific pathways. We further show that the predicted gene expression and pathway activities can help detect cancer subtypes and aid in treatment decision-makings. Given the growing interest in applying SRT in cancer genomics, we believe HOPE2Net holds the potential in identifying biomarkers from direct screenings of tissue histology images, which may be implemented in clinical studies for cancer diagnoses and decision-making processes. Citation Format: Kenong Su, Minxing Pang, Mingyao Li. HOPE2Net: Integrating histological features and position embeddings in spatially resolved transcriptomics to predict gene expression and pathway activities from histology images in tumors [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2022; 2022 Apr 8-13. Philadelphia (PA): AACR; Cancer Res 2022;82(12_Suppl):Abstract nr 1218.