Identifying strengths and weaknesses of methods for computational network inference from single-cell RNA-seq data

Abstract

Single-cell RNA-sequencing (scRNA-seq) offers unparalleled insight into the transcriptional programs of different cellular states by measuring the transcriptome of thousands of individual cells. An emerging problem in the analysis of scRNA-seq is the inference of transcriptional gene regulatory networks and a number of methods with different learning frameworks have been developed to address this problem. Here, we present an expanded benchmarking study of eleven recent network inference methods on six published scRNA-seq datasets in human, mouse, and yeast considering different types of gold standard networks and evaluation metrics. We evaluate methods based on their computing requirements as well as on their ability to recover the network structure. We find that, while most methods have a modest recovery of experimentally derived interactions based on global metrics such as Area Under the Precision Recall curve, methods are able to capture targets of regulators that are relevant to the system under study. Among the top performing methods that use only expression were SCENIC, PIDC, MERLIN or Correlation. Addition of prior biological knowledge and the estimation of transcription factor activities resulted in the best overall performance with the Inferelator and MERLIN methods that use prior knowledge outperforming methods that use expression alone. We found that imputation for network inference did not improve network inference accuracy and could be detrimental. Comparisons of inferred networks for comparable bulk conditions showed that the networks inferred from scRNA-seq datasets are often better or at par with the networks inferred from bulk datasets. Our analysis should be beneficial in selecting methods for network inference. At the same time, this highlights the need for improved methods and better gold standards for regulatory network inference from scRNAseq datasets.

Theoretical and computational tools to model multistable gene regulatory networks

“…The output of a scRNA-seq experiment can be intuitively understood as a count matrix where the rows identify individual cells, and the columns identify the genes in the cell's genome. While this data can be very useful to investigate the gene expression patterns associated to different types of cells, recently several models have been proposed to infer the regulation between genes [82,83]. Here, we briefly introduce the topic and focus on the emerging mathematical challenges.…”

“…For example, in a scenario of multistability cells at different points of a differentiation trajectory coexist at the same physical time while exhibiting different pst values. Interested readers can find more in-depth description of GRN inference methods and their application to biological datasets in topical reviews [82,83].…”

MINI‐AC: inference of plant gene regulatory networks using bulk or single‐cell accessible chromatin profiles

“…While these methods have been successfully used to elucidate regulatory mechanisms (Banf & Rhee, 2017; Haque et al., 2019; Huang et al., 2018; Zhou et al., 2020), they suffer from many false‐positive predictions as no evidence of physical interaction between TF and target gene's regulatory DNA is considered (Banf & Rhee, 2017; Gardner & Faith, 2005; Marbach, Costello, et al., 2012). The integration of TFBS information can further improve GRN predictions (Aibar et al., 2017; Ferrari et al., 2022; Marbach, Roy, et al., 2012; McCalla et al., 2023), but simply mapping TFBS to the non‐coding sequences flanking genes comes with a high rate of false positives. TF motifs are short and degenerate, resulting in low specificity to identify functional TF binding events.…”

NetREm Network Regression Embeddings reveal cell-type transcription factor coordination for gene regulation

“…Despite PPINs having some incorrect PPIs (False Positives (FPs)) and being largely incomplete (Kotlyar et. al 2022), it is helpful to integrate TF-TF PPINs as prior info (McCalla et al 2023;Li and Jackson 2015;Ghanbari et al 2015;Imoto et al 2003;Mukherjee and Speed 2008). This improves NetREm's predicted GRNs from expression data.…”

“…al 2019) that reprograms mouse embryonic fibroblasts to embryonic-like induced pluripotent stem cells. 4: We use normalized data (McCalla et al 2023) from (Shalek et. al 2014) for >1.7k primary bone marrow DCs.…”