A systematic evaluation of single-cell RNA-sequencing imputation methods

Abstract

Background: The rapid development of single-cell RNA-sequencing (scRNA-seq) technologies has led to the emergence of many methods for removing systematic technical noises, including imputation methods, which aim to address the increased sparsity observed in single-cell data. Although many imputation methods have been developed, there is no consensus on how methods compare to each other. Results: Here, we perform a systematic evaluation of 18 scRNA-seq imputation methods to assess their accuracy and usability. We benchmark these methods in terms of the similarity between imputed cell profiles and bulk samples and whether these methods recover relevant biological signals or introduce spurious noise in downstream differential expression, unsupervised clustering, and pseudotemporal trajectory analyses, as well as their computational run time, memory usage, and scalability. Methods are evaluated using data from both cell lines and tissues and from both plateand droplet-based single-cell platforms. Conclusions: We found that the majority of scRNA-seq imputation methods outperformed no imputation in recovering gene expression observed in bulk RNA-seq. However, the majority of the methods did not improve performance in downstream analyses compared to no imputation, in particular for clustering and trajectory analysis, and thus should be used with caution. In addition, we found substantial variability in the performance of the methods within each evaluation aspect. Overall, MAGIC, kNN-smoothing, and SAVER were found to outperform the other methods most consistently.

G2S3: A gene graph-based imputation method for single-cell RNA sequencing data

“…This is likely due to the fact that a small cluster of NP cells were mixed with H1 cells after imputation by MAGIC ( Fig 3 ), resulting in compromised performance in marker gene identification. Our results were largely consistent with a previous evaluation of imputation methods in identifying differentially expressed genes using Fluidigm C1 data [ 33 ]. No genes achieved significance in the imputed data by SAUCIE, so the result of SAUCIE could not be shown.…”

Benchmarking algorithms for pathway activity transformation of single-cell RNA-seq data

“…To evaluate ability of these tools on meaningfully extracting transcriptional heterogeneity, we assessed that the cell-type-specific obtained from PAS should be retained in dimensional reductional space and could assign cells into cell populations through unsupervised clustering or supervised classification [20] , [24] , [26] , [32] , [33] , [34] , [35] , [36] , [37] , [38] , [39] , [40] . Therefore, the accuracy of PAS transformation methods were assessed by three methods of dimensional reduction, clustering and cell type annotation.…”

Benchmarking scRNA-seq imputation tools with respect to network inference highlights deficits in performance at high levels of sparsity

“…In the past years, many single-cell specific imputation tools have already been published [15, 11, 27] and their evaluation showed that denoising sparse simulated data can, for example, help to reobtain original cell clusters and time-course patterns [10]. Because of the rapid increase of published imputation methods, several review articles [20, 7] and benchmarking analysis [40, 25] have been published in recent months, also investigating specific fields within the downstream analysis realm, such as differential gene expression [14]. Unsupervised clustering of cells represents another common downstream tool in the analysis of scRNA-seq data which allows for later cell type classification.…”