Benchmarking UMI-based single-cell RNA-seq preprocessing workflows

Abstract

Background Single-cell RNA-sequencing (scRNA-seq) technologies and associated analysis methods have rapidly developed in recent years. This includes preprocessing methods, which assign sequencing reads to genes to create count matrices for downstream analysis. While several packaged preprocessing workflows have been developed to provide users with convenient tools for handling this process, how they compare to one another and how they influence downstream analysis have not been well studied. Results Here, we systematically benchmark the performance of 10 end-to-end preprocessing workflows (Cell Ranger, Optimus, salmon alevin, alevin-fry, kallisto bustools, dropSeqPipe, scPipe, zUMIs, celseq2, and scruff) using datasets yielding different biological complexity levels generated by CEL-Seq2 and 10x Chromium platforms. We compare these workflows in terms of their quantification properties directly and their impact on normalization and clustering by evaluating the performance of different method combinations. While the scRNA-seq preprocessing workflows compared vary in their detection and quantification of genes across datasets, after downstream analysis with performant normalization and clustering methods, almost all combinations produce clustering results that agree well with the known cell type labels that provided the ground truth in our analysis. Conclusions In summary, the choice of preprocessing method was found to be less important than other steps in the scRNA-seq analysis process. Our study comprehensively compares common scRNA-seq preprocessing workflows and summarizes their characteristics to guide workflow users.

Alevin-fry unlocks rapid, accurate and memory-frugal quantification of single-cell RNA-seq data

“…As sc/snRNA-seq technologies continue to rapidly develop, improving methods used to analyze the resulting data will require ongoing benchmarking of methods to identify the strengths of existing techniques and areas for improvement in future approaches. For example, a recent study by You et al ( 26 ) evaluated many different pipelines for the preprocessing of UMI-based scRNA-seq data. Concordant with the current paper, You et al ( 26 ) found ’s performance to be excellent, both computationally and in terms of the accuracy and robustness of the resulting counts.…”

“…For example, a recent study by You et al ( 26 ) evaluated many different pipelines for the preprocessing of UMI-based scRNA-seq data. Concordant with the current paper, You et al ( 26 ) found ’s performance to be excellent, both computationally and in terms of the accuracy and robustness of the resulting counts. However, they report that — at least when using pseudoalignment with structural constraints — and demonstrate a left-skew in the count distribution of pseudogenes and therefore may underestimate the abundance of transcripts labeled with this biotype.…”

“…While provides an efficient and flexible framework for processing many types of sc/snRNA-seq data, some current implementation limitations, and benchmarking studies such as that performed by You et al ( 26 ), motivate future work. For example, the existing mapping and UMI resolution algorithms are likely not well-suited to long-read scRNA-seq data, though we would like to support such protocols in the future.…”

Depth normalization for single-cell genomics count data

“…While many methods have been proposed for single-cell RNA-seq normalization (Cole et al 2019; Tian et al 2019; You et al 2021; Lytal, Ran, and An 2020; Borella et al 2021; Ahlmann-Eltze and Huber 2021; Breda, Zavolan, and van Nimwegen 2021), the approach of equalizing depth for all cells, often to a “size factor” such as ten thousand (CP10k) or one million (CPM), followed by the application of a variance stabilizing transform like log plus one (log1p) is most popular. These methods are implemented in the widely used Seurat 1 and Scanpy (Wolf, Angerer, and Theis 2018) programs, but they do not explicitly model cell depth as a covariate.…”

Inferring cellular and molecular processes in single-cell data with non-negative matrix factorization using Python, R, and GenePattern Notebook implementations of CoGAPS

“…We acknowledge that these first steps must often vary greatly depending on the biological context, and invite the user to validate optimized custom preprocessing workflows for that context. Comprehensive reviews of preprocessing pipelines for scRNA-seq data have been previously published [43][44][45] .…”