Benchmarking atlas-level data integration in single-cell genomics

Abstract

Single-cell atlases often include samples that span locations, laboratories and conditions, leading to complex, nested batch effects in data. Thus, joint analysis of atlas datasets requires reliable data integration. To guide integration method choice, we benchmarked 68 method and preprocessing combinations on 85 batches of gene expression, chromatin accessibility and simulation data from 23 publications, altogether representing >1.2 million cells distributed in 13 atlas-level integration tasks. We evaluated methods according to scalability, usability and their ability to remove batch effects while retaining biological variation using 14 evaluation metrics. We show that highly variable gene selection improves the performance of data integration methods, whereas scaling pushes methods to prioritize batch removal over conservation of biological variation. Overall, scANVI, Scanorama, scVI and scGen perform well, particularly on complex integration tasks, while single-cell ATAC-sequencing integration performance is strongly affected by choice of feature space. Our freely available Python module and benchmarking pipeline can identify optimal data integration methods for new data, benchmark new methods and improve method development.

Comprehensive evaluation of noise reduction methods for single-cell RNA sequencing data

“…Recent efforts to benchmark batch correction methods for single-cell RNA-seq all came to the conclusion that no single method emerges as the best performer in every dataset [ 23 , 33 , 34 ]. They also showed the importance of the selection of highly variable genes and the limitation of different types of outputs on the downstream analysis.…”

Single-Cell Sequencing Unveils the Heterogeneity of Nonimmune Cells in Chronic Apical Periodontitis

“…To normalize and scale the single-cell gene expression data, we imported the filtered single cells and the UMI count matrices into the Seurat (v3.1.5) R package (v3.5.2, https://www.R-project.org/ ) ( Satija et al, 2015 ; Butler et al, 2018 ). By the Seurat “FindVariableGenes” function, we determined the highly variable genes (HGVs) across the single cells in each sample ( Luecken et al, 2021 ). Following principal component analysis (PCA), the first 20 principal components were selected for cell clustering and dimension reduction visualized by t-SNE and UMAP plots at a resolution of 0.5 ( Supplementary Figure S3 ).…”

Deep learning shapes single-cell data analysis

“…As DL models for single-cell data analysis have not matured, it may be valuable to run multiple tools to see how they compare. Furthermore, comprehensive single-cell DL benchmarking papers help users choose the best model 8 , 9 .…”