Benchmarking imputation methods for network inference using a novel method of synthetic scRNA-seq data generation

Abstract

Background Single cell RNA-sequencing (scRNA-seq) has very rapidly become the new workhorse of modern biology providing an unprecedented global view on cellular diversity and heterogeneity. In particular, the structure of gene-gene expression correlation contains information on the underlying gene regulatory networks. However, interpretation of scRNA-seq data is challenging due to specific experimental error and biases that are unique to this kind of data including drop-out (or technical zeros). Methods To deal with this problem several methods for imputation of zeros for scRNA-seq have been developed. However, it is not clear how these processing steps affect inference of genetic networks from single cell data. Here, we introduce Biomodelling.jl, a tool for generation of synthetic scRNA-seq data using multiscale modelling of stochastic gene regulatory networks in growing and dividing cells. Results Our tool produces realistic transcription data with a known ground truth network topology that can be used to benchmark different approaches for gene regulatory network inference. Using this tool we investigate the impact of different imputation methods on the performance of several network inference algorithms. Conclusions Biomodelling.jl provides a versatile and useful tool for future development and benchmarking of network inference approaches using scRNA-seq data.

A systematic comparison of computational methods for expression forecasting

“…Prior benchmarks of gene regulatory network inference in mammals have mostly focused on structure recovery (Chevalley, Roohani, Mehrjou, Leskovec, & Schwab, 2022;Lasri, Shahrezaei, & Sturrock, 2022;Marbach et al, 2012;McCalla et al, 2023;Pratapa, Jalihal, Law, Bharadwaj, & Murali, 2020;Saint-Antoine & Singh, 2023), which is particularly challenging (Badia-I-Mompel et al, 2023). Some structure benchmarks use protein-protein interactions, which does not establish a causal effect on transcription.…”

Bayesian model discovery for reverse-engineering biochemical networks from data

“…GENIE3, an acronym for GEne Network Inference with Ensemble of trees, was the best performer at the DREAM4/DREAM5 network challenges [38], and though developed more than a decade ago, it still remains a favoured method for this kind of inference [22, 26]. The algorithm uses an ensemble of decision trees to model the expression of a gene as a function of the expression of other genes.…”

A Framework for Comparison and Assessment of Synthetic RNA-Seq Data

“…With the increased interest in developing new methods for analysis of bulk and scRNA-seq data, the need for evaluating methodologies and benchmarking the implementations of the associated algorithms also increases. Some of the most popular metrics used for such benchmark performance evaluation are the false discovery rate (FDR) and sensitivity [ 40 ], as well as classification error estimation [ 41 ]; additionally, clustering [ 42 ] and network inference accuracy [ 43 ] are important areas of interest. Using real datasets for these types of performance evaluation studies is not reliable because one lacks knowledge of ground truth related to gene expression levels, their differences between populations, their interactions, or even their class label.…”