Stability Indicators in Network Reconstruction

Abstract

The number of available algorithms to infer a biological network from a dataset of high-throughput measurements is overwhelming and keeps growing. However, evaluating their performance is unfeasible unless a ‘gold standard’ is available to measure how close the reconstructed network is to the ground truth. One measure of this is the stability of these predictions to data resampling approaches. We introduce NetSI, a family of Network Stability Indicators, to assess quantitatively the stability of a reconstructed network in terms of inference variability due to data subsampling. In order to evaluate network stability, the main NetSI methods use a global/local network metric in combination with a resampling (bootstrap or cross-validation) procedure. In addition, we provide two normalized variability scores over data resampling to measure edge weight stability and node degree stability, and then introduce a stability ranking for edges and nodes. A complete implementation of the NetSI indicators, including the Hamming-Ipsen-Mikhailov (HIM) network distance adopted in this paper is available with the R package nettools. We demonstrate the use of the NetSI family by measuring network stability on four datasets against alternative network reconstruction methods. First, the effect of sample size on stability of inferred networks is studied in a gold standard framework on yeast-like data from the Gene Net Weaver simulator. We also consider the impact of varying modularity on a set of structurally different networks (50 nodes, from 2 to 10 modules), and then of complex feature covariance structure, showing the different behaviours of standard reconstruction methods based on Pearson correlation, Maximum Information Coefficient (MIC) and False Discovery Rate (FDR) strategy. Finally, we demonstrate a strong combined effect of different reconstruction methods and phenotype subgroups on a hepatocellular carcinoma miRNA microarray dataset (240 subjects), and we validate the analysis on a second dataset (166 subjects) with good reproducibility.

ReNette: a web-infrastructure for reproducible network analysis

“…The inference and the network stability procedures return the adjacency matrix for each input file and selected reconstruction method as a CSV file. The network stability procedure outputs also the netSI Filosi et al (2014) indicators in 3 different CSV files. The network distance returns, for each selected method, a matrix with the pairwise distances between all possible pairs of input files.…”

“…The stability framework implemented here is based on Filosi et al (2014). The included inference methods are CLR, ARACNE, WGCNA, bicor and TOM Filosi et al (2014), MINE and DTWMIC Albanese et al (2013); Riccadonna et al (2012). For WGCNA, bicor and MINE a False Discovery Rate controlled version is also implemented Filosi et al (2014).…”

“…The included inference methods are CLR, ARACNE, WGCNA, bicor and TOM Filosi et al (2014), MINE and DTWMIC Albanese et al (2013); Riccadonna et al (2012). For WGCNA, bicor and MINE a False Discovery Rate controlled version is also implemented Filosi et al (2014). The available network metrics are the Hammingdistance, the Ipsen-Mikhailovdistance and the novel HIM product metric previously proposed in Jurman et al (2013).…”

Weighted Network Estimation by the Use of Topological Graph Metrics

“…Typical applications include predicting the appearance of future connections in social [10], [15] or biological [16] networks. The network reconstruction problem deals with composing networks that satisfy specific properties [17]- [20]. This can be particularly useful when building null models for hypothesis testing.…”

A null model for Pearson coexpression networks

“… 1 The HIM distance [69, 70] is a metric between networks having the same nodes, ranging between 0 for identical networks and 1, attained comparing the clique with the empty graph.…”