Leveraging information in spatial transcriptomics to predict super-resolution gene expression from histology images in tumors

Abstract

Recent developments in spatial transcriptomics (ST) technologies have enabled the profiling of transcriptome-wide gene expression while retaining the location information of measured genes within tissues. Moreover, the corresponding high-resolution hematoxylin and eosin-stained histology images are readily available for the ST tissue sections. Since histology images are easy to obtain, it is desirable to leverage information learned from ST to predict gene expression for tissue sections where only histology images are available. Here we present HisToGene, a deep learning model for gene expression prediction from histology images. To account for the spatial dependency of measured spots, HisToGene adopts Vision Transformer, a state-of-the-art method for image recognition. The well-trained HisToGene model can also predict super-resolution gene expression. Through evaluations on 32 HER2+ breast cancer samples with 9,612 spots and 785 genes, we show that HisToGene accurately predicts gene expression and outperforms ST-Net both in gene expression prediction and clustering tissue regions using the predicted expression. We further show that the predicted super-resolution gene expression also leads to higher clustering accuracy than observed gene expression. Gene expression predicted from HisToGene enables researchers to generate virtual transcriptomics data at scale and can help elucidate the molecular signatures of tissues.

Recent advances in spatially resolved transcriptomics: challenges and opportunities

“…In addition, HisToGene adopts Vision Transformer for image recognition and predicts super-resolution gene expression. Using the same dataset from ST-Net as a training set, HisToGene outperformed ST-Net in both gene expression prediction and clustering tissue regions accuracy ( 45 ). stLearn also integrates three types of datasets, including spatial dimensionality, tissue morphology, and genome-wide transcriptional profile using a deep learning network model ( 46 ) and predict cell type clustering, intercellular interaction, and reconstruction of spatial transition gradients, which were all successfully conducted with brain and breast cancer datasets.…”

Spatial Transcriptomics Prediction from Histology jointly through Transformer and Graph Neural Networks

“…Spatial region detection on the HER2+ dataset using gene expressions obtained by all methods, where the observed represents the direct use of sequenced gene expressions, and the GT represents the ground truth labels from the pathology annotations. The results of HisToGene and ST-Net were directly obtained from the reference [27]. …”

“…To solve this issue, HisToGene [27] is designed to include spot spatial relations through the Vision Transformer (VIT) [31], where VIT uses a self-attention mechanism to capture the relationships between spots [32]. The attention mechanism has shown stable performance on many tasks such as registration [33] and segmentation [34].…”

Deep topographic proteomics of a human brain tumour

“…This bulk property means that the data still suffers from missing values which can confound spatial pathway level analysis. Future approaches are bound to use systematic spatial proteomic analysis, possibly compromising spatial resolution but incorporating an element of machine learning to use orthogonal higher resolution omics and imaging data to infer protein abundance towards individual cell resolution, as can be done on spatial transcriptomics data [58][59][60][61] . In addition, with the detection of LCM-based and cell-type resolved deep proteomes, these data will be highly complementary to current imaging technologies and increase the understanding of spatially resolved biological and pathological processes at the molecular level 29,32,57 .…”