Identifying spatial domain by adapting transcriptomics with histology through contrastive learning

Abstract

Recent advances in spatial transcriptomics have enabled measurements of gene expression at cell/spot resolution meanwhile retaining both the spatial information and the histology images of the tissues. Accurately identifying the spatial domains of spots is a vital step for various downstream tasks in spatial transcriptomics analysis. To remove noises in gene expression, several methods have been developed to combine histopathological images for data analysis of spatial transcriptomics. However, these methods either use the image only for the spatial relations for spots, or individually learn the embeddings of the gene expression and image without fully coupling the information. Here, we propose a novel method ConGI to accurately exploit spatial domains by adapting gene expression with histopathological images through contrastive learning. Specifically, we designed three contrastive loss functions within and between two modalities (the gene expression and image data) to learn the common representations. The learned representations are then used to cluster the spatial domains on both tumor and normal spatial transcriptomics datasets. ConGI was shown to outperform existing methods for the spatial domain identification. In addition, the learned representations have also been shown powerful for various downstream tasks, including trajectory inference, clustering, and visualization.

Spatial domain detection using contrastive self-supervised learning for spatial multi-omics technologies

“…However, for some proteins, such as Abeta, expression may be disconnected on the spot level, especially in late stages of progressive diseases, potentially making it challenging for Proust to accurately identify all regions where the protein occurs, as individual spots may absorb dissimilar neighboring information during model training. To address this issue, a potential solution is to enhance the graph structure with weighted edges based on the similarity of spot-level protein information or to incorporate inter-modality contrastive learning to maximize the mutual information between gene expression and proteomics [23, 34]. We are also interested in exploring the use of statistical inference here as a way to explore the uncertainty of the predicted spatial domains.…”

“…A second approach is to continue with only one omic data modality, but to incorporate spatial information to account for the correlation of molecular information between the spatial coordinates. Some examples of these methods include (i) unsupervised learning approaches (BayesSpace [18], Giotto [19], STAGATE [20], CCST [21]) and (ii) self-supervised learning approaches (GraphST [15], SpaceFlow [22], ConGI [23], CAST [24]). In particular, the methods using contrastive self-supervised learning aim to maximize the similarity between adjacent spatial coordinates and dissimilarity between non-adjacent spatial coordinates, while also showing great promise in their ability to detect discrete spatial domains using only one data modality.…”

Uncover spatially informed shared variations for single-cell spatial transcriptomics with STew

“…Although methods using deep learning-based models have been developed to decipher spatial domains by combining histological images with gene expression data (Pham et al . 2020; Zeng et al . 2023), several potential drawbacks exist.…”

Assessment of Emerging Pretraining Strategies in Interpretable Multimodal Deep Learning for Cancer Prognostication

“…A contrastive cross-modal model encoding image patches and spatial transcriptomic profiles was created, similar to the model implemented by Zeng et al 21 . Input images patches of size 224x224 were encoded into embeddings of size 512 units, using the feature extraction portion of a CNN initialized with weights initialized from the ResNet model trained by Ciga et al Spatial transcriptomics profiles containing expression of the most spatially variable 1000 genes across Visium slides, selected to avoid overfitting on genes with imprecise expression, were encoded with three standard fully connected (FC) layers of size 512.…”