scTransSort: Transformers for Intelligent Annotation of Cell Types by Gene Embeddings

Abstract

Single-cell transcriptomics is rapidly advancing our understanding of the composition of complex tissues and biological cells, and single-cell RNA sequencing (scRNA-seq) holds great potential for identifying and characterizing the cell composition of complex tissues. Cell type identification by analyzing scRNA-seq data is mostly limited by time-consuming and irreproducible manual annotation. As scRNA-seq technology scales to thousands of cells per experiment, the exponential increase in the number of cell samples makes manual annotation more difficult. On the other hand, the sparsity of gene transcriptome data remains a major challenge. This paper applied the idea of the transformer to single-cell classification tasks based on scRNA-seq data. We propose scTransSort, a cell-type annotation method pretrained with single-cell transcriptomics data. The scTransSort incorporates a method of representing genes as gene expression embedding blocks to reduce the sparsity of data used for cell type identification and reduce the computational complexity. The feature of scTransSort is that its implementation of intelligent information extraction for unordered data, automatically extracting valid features of cell types without the need for manually labeled features and additional references. In experiments on cells from 35 human and 26 mouse tissues, scTransSort successfully elucidated its high accuracy and high performance for cell type identification, and demonstrated its own high robustness and generalization ability.

New perspectives on biology, disease progression, and therapy response of head and neck cancer gained from single cell RNA sequencing and spatial transcriptomics

“…CNVs are a defining feature of tumor cells and can be detected by CAISIC [41], CaSpER [42], or InferCNV [43], which assumes that genes located in regions with CNVs will also display alterations in their RNA expression levels. Several tools are available for cell type annotation [44][45][46][47][48][49][50][51][52][53][54][55][56][57]. SingleR [58] and CellAssign [59] facilitated cell type identification based on reference datasets from pure cell types and known marker gene sets, respectively.…”

Tracing unknown tumor origins with a biological-pathway-based transformer model

“…20 , 25 , 26 , 27 The inclusion of the multiheaded attention mechanism in transformer enables the intricate extraction of gene-specific features within the transcriptome, 28 which allows the transformer to effectively analyze gene interdependencies in high-dimensional, sparse biological data. 29 As a result, transformer has pushed the boundaries in numerous tasks related to gene expression profiles, including cancer subtype classification, 30 phenotype prediction, 31 cell-type annotation, 32 and multimodal cancer data integration. 33 To improve the interpretability of the transformer applied to biomedical tasks, several studies have sought to integrate prior biological knowledge into the model.…”