Self-supervised contrastive learning for integrative single cell RNA-seq data analysis

Han, Wenkai; Cheng, Yuqi; Chen, Jiayang; Zhong, Hai; Hu, Zhihang; Chen, Siyuan; Zong, Licheng; Liang, Hong; Chan, Ting‐Fung; King, Irwin; Gao, Xin; Li, Yu

doi:10.1093/bib/bbac377

Cited by 44 publications

(25 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, our benchmarking of SSL methods revealed the sensitivity to the choice of pre-training strategy. While contrastive methods have shown efficacy in other domains 41,43,44 and specialized in smaller scales in SCG 17,18,[20][21][22][23][24][25][26] , our study found that standard contrastive approaches did not yield as promising results for diverse, large-scale SCG tasks. This result highlights the challenges of applying these methods as generalizable pretext tasks for single-cell data.…”

Section: A Tailored Pre-training Strategy Leads To High Zero-shot Per...mentioning

confidence: 65%

Delineating the Effective Use of Self-Supervised Learning in Single-Cell Genomics

Richter,

Bahrami,

Xia

et al. 2024

Preprint

View full text Add to dashboard Cite

Self-supervised learning (SSL) has emerged as a powerful method for extracting meaningful representations from vast, unlabeled datasets, already transforming computer vision and natural language processing. Similarly, in single-cell genomics (SCG), representation learning is well-recognized for offering insights into complex biological data, even more so by the advent of early foundation model approaches. However, despite these advancements, identifying scenarios in SCG where SSL outperforms traditional supervised or unsupervised learning methods remains a nuanced challenge. Furthermore, selecting the most effective pretext tasks within the SSL framework for SCG is a critical yet unresolved question. Here, we address this gap by adapting and benchmarking SSL techniques in SCG, including masked autoencoders with multiple masking strategies and contrastive learning approaches. Trained on over 20 million cells, this study rigorously examines multiple downstream tasks, including cell type prediction, gene expression reconstruction, cross-modality prediction, and data integration. Our empirical analyses underscore the nuanced role of SSL, namely in transfer learning scenarios leveraging auxiliary data or analyzing novel datasets. Masked autoencoders excel over contrastive methods in SCG, diverging from computer vision trends. Moreover, our findings reveal notable capabilities of SSL in zero-shot cell type prediction and offer insights into its potential benefits in cross-modality prediction and data integration. In summary, we study the application of SSL in SCG, minimizing model bias through simple, fully connected networks, and benchmark SSL's utility across key representation learning scenarios.

show abstract

Section: A Tailored Pre-training Strategy Leads To High Zero-shot Per...mentioning

confidence: 65%

Delineating the Effective Use of Self-Supervised Learning in Single-Cell Genomics

Richter,

Bahrami,

Xia

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…The cell type labels could refer to biological cell types or the labels of data batches collected from different times or platforms. Following the idea of contrastive learning ( Schroff et al 2015 ; Han et al 2022 ), we employ a contrastive loss in embedding space. It enforces smaller In-Batch distance and larger Between-Batch distance.…”

Section: Methodsmentioning

confidence: 99%

Con-AAE: contrastive cycle adversarial autoencoders for single-cell multi-omics alignment and integration

Wang

et al. 2023

Bioinformatics

View full text Add to dashboard Cite

Motivation We have entered the multi-omics era and can measure cells from different aspects. Hence, we can get a more comprehensive view by integrating or matching data from different spaces corresponding to the same object. However, it is particularly challenging in the single-cell multi-omics scenario because such data are very sparse with extremely high dimensions. Though some techniques can be used to measure scATAC-seq and scRNA-seq simultaneously, the data are usually highly noisy due to the limitations of the experimental environment. Results To promote single-cell multi-omics research, we overcome the above challenges, proposing a novel framework, contrastive cycle adversarial autoencoders, which can align and integrate single-cell RNA-seq data and single-cell ATAC-seq data. Con-AAE can efficiently map the above data with high sparsity and noise from different spaces to a coordinated subspace, where alignment and integration tasks can be easier. We demonstrate its advantages on several datasets. Availability Zenodo link: https://zenodo.org/badge/latestdoi/368779433 github: https://github.com/kakarotcq/Con-AAE. Supplementary information Supplementary data are available at Bioinformatics online.

show abstract

“…During the training phase, after feature selection and term frequency-inverse document frequency (TF-IDF) transformation (see more details in Section 5), we utilize contrastive learning to learn latent representations of the training set, which have yielded state-of-theart results on various tasks in single-cell data analysis [28][29][30]. Specifically, we utilize the multi-layer perceptron (MLP) module within the contrastive learning framework, as suggested by recent studies [31][32][33].…”

Section: The Overview Of Rainbowmentioning

confidence: 99%

Accurate cell type annotation for single‐cell chromatin accessibility data via contrastive learning and reference guidance

Li,

Tang,

Wang

et al. 2024

Quant. Biol.

View full text Add to dashboard Cite

Recent advances in single‐cell chromatin accessibility sequencing (scCAS) technologies have resulted in new insights into the characterization of epigenomic heterogeneity and have increased the need for automatic cell type annotation. However, existing automatic annotation methods for scCAS data fail to incorporate the reference data and neglect novel cell types, which only exist in a test set. Here, we propose RAINBOW, a reference‐guided automatic annotation method based on the contrastive learning framework, which is capable of effectively identifying novel cell types in a test set. By utilizing contrastive learning and incorporating reference data, RAINBOW can effectively characterize the heterogeneity of cell types, thereby facilitating more accurate annotation. With extensive experiments on multiple scCAS datasets, we show the advantages of RAINBOW over state‐of‐the‐art methods in known and novel cell type annotation. We also verify the effectiveness of incorporating reference data during the training process. In addition, we demonstrate the robustness of RAINBOW to data sparsity and number of cell types. Furthermore, RAINBOW provides superior performance in newly sequenced data and can reveal biological implication in downstream analyses. All the results demonstrate the superior performance of RAINBOW in cell type annotation for scCAS data. We anticipate that RAINBOW will offer essential guidance and great assistance in scCAS data analysis. The source codes are available at the GitHub website (BioX‐NKU/RAINBOW).

show abstract

Self-supervised contrastive learning for integrative single cell RNA-seq data analysis

Cited by 44 publications

References 59 publications

Delineating the Effective Use of Self-Supervised Learning in Single-Cell Genomics

Delineating the Effective Use of Self-Supervised Learning in Single-Cell Genomics

Con-AAE: contrastive cycle adversarial autoencoders for single-cell multi-omics alignment and integration

Accurate cell type annotation for single‐cell chromatin accessibility data via contrastive learning and reference guidance

Contact Info

Product

Resources

About