Quantifying the multi-scale performance of network inference algorithms

Abstract

Graphical models are widely used to study complex multivariate biological systems. Network inference algorithms aim to reverse-engineer such models from noisy experimental data. It is common to assess such algorithms using techniques from classifier analysis. These metrics, based on ability to correctly infer individual edges, possess a number of appealing features including invariance to rank-preserving transformation. However, regulation in biological systems occurs on multiple scales and existing metrics do not take into account the correctness of higher-order network structure.In this paper novel performance scores are presented that share the appealing properties of existing scores, whilst capturing ability to uncover regulation on multiple scales. Theoretical results confirm that performance of a network inference algorithm depends crucially on the scale at which inferences are to be made; in particular strong local performance does not guarantee accurate reconstruction of higher-order topology. Applying these scores to a large corpus of data from the DREAM5 challenge, we undertake a data-driven assessment of estimator performance. We find that the "wisdom of crowds" network, that demonstrated superior local performance in the DREAM5 challenge, is also among the best performing methodologies for inference of regulation on multiple length scales.MATLAB R2013b code net_assess is provided as Supplement.

Gene Regulatory Network Inference from Single-Cell Data Using Multivariate Information Measures

“…Intricate networks of transcriptional activators and repressors have evolved to regulate the spatial and temporal expression of genes, enabling organisms to adjust transcription levels in response to environmental, developmental, and physiological cues ( Trapnell et al., 2014 , Harrington et al., 2014 , Rue and Martinez Arias, 2015 , Moris et al., 2016 , Gouti et al., 2015 , Göttgens, 2015 ). Elucidating the structure of such gene regulatory networks (GRNs) has been a central goal of much recent systems biology research ( De Smet and Marchal, 2010 , Oates and Mukherjee, 2012 , Thorne and Stumpf, 2012 , Thorne et al., 2013 , Siegenthaler and Gunawan, 2014 , Oates et al., 2014 , Huang and Zi, 2014 , Young et al., 2014 ), and it is now becoming a pivotal stepping stone in dissecting the molecular contributions of complex diseases ( Boyle et al., 2017 ).…”

Bayesian state space models for dynamic genetic network construction across multiple tissues

“…In this way, we can speed up inferring the medium to large size interaction networks that may incorporate other environmental covariates for more interpretable results. Moreover, the recently developed other prior distributions, such as double exponential priors, spike and slab priors with Markov random field, non-conjugate Dirichlet process algorithms, new model selection criterion such as Bayesian predictive information criterion (BPIC), and multi-scale network inference algorithms with multi-scale performance scores for handling higher-order network structures could be explored and compared for learning the robustness of modeling the large dynamic genomic network relatedness structure and complex multivariate biological systems from a predictive view point (Akutsu et al, 2000;Lunn et al, 2000;Spiegelhalter et al, 2002;Segal et al, 2003;Segul et al, 2003;Shannon et al, 2003;Ando, 2007;Carvalho and Scott, 2009;Marbach et al, 2009;Finegold and Drton, 2014;Oates et al, 2014;Spiegelhalter et al, 2014).…”

How to deal with parameters for whole-cell modelling

“…The structure of the model-in terms of the mathematical representation of the function f(Y; u, t) which describes the system components and relationships-is defined according to our current knowledge, perhaps in combination with datadriven network inference techniques that aim to learn the likely structure of a system from observations of its variables [40][41][42][43][44][45][46]. However, we also need to obtain suitable estimates for the parameters, either from experimentally determined values or by using statistical approaches to estimate (or infer) these values by fitting model simulations to observed data.…”