Benchmarking strategies for cross-species integration of single-cell RNA sequencing data

Abstract

The emergence of multiple species cell atlases has opened the opportunity to investigate evolutionary relationships at the cell type level and understand similarities and differences between cell types across species. Cross-species integration of single-cell RNA-sequencing data has been a key approach to comparing cell types between species. To robustly identify homologous cell types and species-specific cell populations, it is crucial to obtain reliable and informative integration. However, currently there are neither benchmarks nor guidelines on how to choose among a diversity of integration strategies. To examine the output characteristics of different cross-species integration strategies, we benchmarked 25 combinations of gene homology mapping methods and integration algorithms in a variety of biological settings. We examine each strategy's capability to perform species-mixing of known homologous cell types and preserve biological heterogeneity using 10 metrics, among which we developed a new biology conservation metric to address the maintenance of cell type distinguishability. Overall, scVI, SeuratV4 methods and SAMap achieve a balance between species-mixing and biology conservation. For evolutionarily distant species, including in-paralogs and using nearest-neighbour based methods is beneficial. SAMap outperforms when integrating whole-body atlases and can prioritize identity-defining similarities between homologous cell types in complex circumstances. We provide our freely available cross-species integration and assessment pipeline to help identify the optimum strategy to analyse new data and develop new algorithms.

Towards Universal Cell Embeddings: Integrating Single-cell RNA-seq Datasets across Species with SATURN

“…We evaluate performance by measuring how well labels can be transferred from zebrafish to frog. In particular, we first integrate the datasets using SATURN and then use the cell-type annotations of cells from a reference species, zebrafish, to train a logistic classifier to predict cell types [29] (Supplementary Note 3). The classifier's performance is then tested on the embeddings of the query species, frog (Fig.…”

Integrating single-cell RNA-seq datasets with substantial batch effects

“…Additionally, while we heere did not observe a benefit in more complex orthologue mapping for cross-species integration, this may be of greater importance when integrating more evolutionary divergent species. Thus, future work could explore the effect of different gene mapping strategies, both as part of data preprocessing 19 as well as in the model internally, such as by enabling flexible gene relationships 5,30 or using gene embeddings 49,52 .…”

“…These methods enable good integration of batch effects arising by processing similar samples in different laboratories, however, they do not enable sufficient integration when differences between datasets are more substantial due to datasets originating from distinct biological or technical "systems", such as multiple species or sequencing technologies (e.g. cell-nuclei) [16][17][18] (Figure 1a,b). To improve performance different approaches for tuning and increasing batch correction have been proposed, such as Kullback-Leibler divergence (KL) regularization strength tuning 22 , latent space adversarial learning [23][24][25] , and latent space cycle-consistency that was previously used specifically for multi-omic integration [26][27][28] , and for increasing biological preservation the use of the multimodal variational mixture of posteriors (VampPrior) 29 as the prior for the latent space was proposed 30 .…”

“…However, while cVAE-based and other non-deep-learning methods enable good integration of batch effects caused by processing similar samples in different laboratories, they do not enable sufficient integration when differences between datasets are more substantial due to datasets originating from distinct biological or technical "systems", such as multiple species or sequencing technologies (e.g. cell-nuclei) [19][20][21] (Figure 1a,b), as demonstrated later. To enable more comprehensive single-cell atlasing efforts that will integrate diverse samples with stronger batch effects, it is thus vital to further improve the performance of the commonly used cVAE-based integration models.…”

Multi-species atlas resolves an axolotl limb development and regeneration paradox

“…To test this, we first collected publicly available scRNA-Seq datasets of developing limbs from axolotls ( Ambystoma mexicanum ) 23 and, species with their developmental stages that are documented to have an AER: humans ( Homo sapiens ) 24, 25 , mice ( Mus musculus ) 26, 27 , chickens ( Gallus gallus ) 28 , and frogs ( Xenopus laevis ) 23, 29 ) ( Figures S1 and S2 and Supplementary Table 1) . Then, using Seurat integration, which has been shown to integrate cross-species datasets with high accuracy 30 , we established a multi-species limb atlas that in total contains 50,248 representative cells from all samples ( Figures 1B and S3 ). Coarse lineage annotation detected various mesodermal and ectodermal populations in the multi-species limb atlas, and finer annotation captured the AER cluster ( Figures 1C and S3 ).…”