Benchmarking scRNA-seq imputation tools with respect to network inference highlights deficits in performance at high levels of sparsity

Abstract

Gene correlation network inference from single-cell transcriptomics data potentially allows to gain unprecendented insights into cell type-specific regulatory programs. ScRNA-seq data is severely affected by dropout, which significantly hampers and restrains current downstream analysis. Although newly developed tools are capable to deal with sparse data, no appropriate single-cell network inference workflow has been established. A potential way to end this deadlock is the application of data imputation methods, which already proofed to be useful in specific contexts of single-cell data analysis, e.g., recovering cell clusters. In order to infer cell-type specific networks, two prerequisites must be met: the identification of cluster-specific cell-types and the network inference itself. Here, we propose a benchmarking framework to investigate both objections. By using suitable reference data with inherent correlation structure, six representative imputation tools and appropriate evaluation measures, we were able to systematically infer the impact of data imputation on network inference. Major network structures were found to be preserved in low dropout data sets. For moderately sparse data sets, DCA was able to recover gene correlation structures, although systematically introducing higher correlation values. No imputation tool was able to recover true signals from high dropout data. However, by using an additional biological data set we could show that cell-cell correlation by means of specific marker gene expression was not compromised through data imputation. Our analysis showed that network inference is feasible for low and moderately sparse data sets by using the unimputed and DCA-prepared data, respectively. High sparsity data, on the other side, still pose a major problem since current imputation techniques are not able to facilitate network inference. The annotation of cluster-specific cell-types as a prerequisite is not hampered by data imputation but their power to restore the deeply hidden correlation structures is still not sufficient enough.

Effect of imputation on gene network reconstruction from single-cell RNA-seq data

“…In agreement with previous studies, we see that imputation may boost gene-gene correlations in a questionable way, thereby introducing FP edges in a network. 23,37 Steinheuer et al 37 evaluated the impact of data imputation on network inference via a gene correlation analysis using simulated data. There, the authors downsampled bulk RNA-seq data, applied imputation methods, and compared the gene module preservation and edge recovery upon imputation.…”

“…In agreement with previous studies, we see that imputation may boost gene-gene correlations in a questionable way, thereby introducing FP edges in a network. 23 , 37 Steinheuer et al. 37 evaluated the impact of data imputation on network inference via a gene correlation analysis using simulated data.…”

“… 23 , 37 Steinheuer et al. 37 evaluated the impact of data imputation on network inference via a gene correlation analysis using simulated data. There, the authors downsampled bulk RNA-seq data, applied imputation methods, and compared the gene module preservation and edge recovery upon imputation.…”

Benchmarking imputation methods for network inference using a novel method of synthetic scRNA-seq data generation

“…does not improve the network inference precision over downsampled data (Supplemental Figures 3(C),(G),(K), 4(C),(G) and 5(C),(G)), unlike what was reported in the aforementioned study where the authors observed that combining SAVER and GENIE3 does improve network inference performance in some cases. In [81], it was reported that low capture efficiencies pose a challenge for imputation and network inference methods and that some imputation methods, namely DCA [23], preserve the gene-gene correlations structure even though false positive correlations are introduced, these findings are consistent with our results (Supplemental Figures 3, 4 and 5) where we found that for low capture efficiencies, regardless of the imputation and network inference method, the network inference precision is poor and we also found that SAVER similar to DCA preserves the gene-gene correlations structure as mentioned above.…”

“…Finally, we compared our results with two recent complementary studies that investigated the impact of imputation on network reconstruction performance [80,81].…”