Reconstructing Networks with Unknown and Heterogeneous Errors

Abstract

The vast majority of network datasets contains errors and omissions, although this is rarely incorporated in traditional network analysis. Recently, an increasing effort has been made to fill this methodological gap by developing network reconstruction approaches based on Bayesian inference. These approaches, however, rely on assumptions of uniform error rates and on direct estimations of the existence of each edge via repeated measurements, something that is currently unavailable for the majority of network data. Here we develop a Bayesian reconstruction approach that lifts these limitations by not only allowing for heterogeneous errors, but also for single edge measurements without direct error estimates. Our approach works by coupling the inference approach with structured generative network models, which enable the correlations between edges to be used as reliable uncertainty estimates. Although our approach is general, we focus on the stochastic block model as the basic generative process, from which efficient nonparametric inference can be performed, and yields a principled method to infer hierarchical community structure from noisy data. We demonstrate the efficacy of our approach with a variety of empirical and artificial networks.

Network Reconstruction and Community Detection from Dynamics

“…We amend this inconsistency in the same manner as in Ref. [34], by adapting the multigraph models to simple graphs in tractable way by generating multigraphs and then collapsing the multiple edges. In other words, if G is a multigraph with entries G ij ∈ N, the collapsed simple graph A(G) has binary entries…”

“…Bayesian network reconstruction -We approach the network reconstruction task similarly to the situation where the network edges are measured directly, but via an uncertain process [33,34]: If D is the measurement of some process that takes place on a network, we can define a posterior distribution for the underlying adjacency matrix A via Bayes' rule,…”

“…The use of the DC-SBM as a prior probability in Eq. 2 is motivated by its ability to inform link prediction in networks where some fraction of edges have not been observed or have been observed erroneously [34,38]. The latent conditional probabilities of edges existing between groups of nodes is learned by the collective observation of many similar edges, and these correlations are leveraged to extrapolate the existence of missing or spurious ones.…”

Reconstruction of plant–pollinator networks from observational data

“…As outlined in the main text, our method relies on a generative network model in which observed visits to plants by pollinators are considered noisy measurements of an unobserved underlying plant-pollinator network. This formulation allows us to frame the task of determining the network structure as a Bayesian inference problem [27][28][29][30] in which the probability of the network having incidence matrix B given a…”

Exploring the solution landscape enables more reliable network community detection

“…The resulting communities form global-scale areas that share similar species called bioregions. We analyzed the community structure with the hierarchical versions of Infomap [21] and the BSBM [35] by generating 1500 partitions with each algorithm. We chose d max = 0.2, which roughly corresponds to partition differences that cover up to 20% of the Earth's surface.…”