Deriving pairwise transfer entropy from network structure and motifs

Abstract

<jats:p>Transfer entropy (TE) is an established method for quantifying directed statistical dependencies in neuroimaging and complex systems datasets. The pairwise (or bivariate) TE from a source to a target node in a network does not depend solely on the local source-target link weight, but on the wider network structure that the link is embedded in. This relationship is studied using a discrete-time linearly coupled Gaussian model, which allows us to derive the TE for each link from the network topology. It is shown analytically that the dependence on the directed link weight is only a first approximation, valid for weak coupling. More generally, the TE increases with the in-degree of the source and decreases with the in-degree of the target, indicating an asymmetry of information transfer between hubs and low-degree nodes. In addition, the TE is directly proportional to weighted motif counts involving common parents or multiple walks from the source to the target, which are more abundant in networks with a high clustering coefficient than in random networks. Our findings also apply to Granger causality, which is equivalent to TE for Gaussian variables. Moreover, similar empirical results on random Boolean networks suggest that the dependence of the TE on the in-degree extends to nonlinear dynamics.</jats:p>

Inferring network properties from time series using transfer entropy and mutual information: Validation of multivariate versus bivariate approaches

“…Nonetheless, the key message is that the modular structure (at the mesoscale level) affects the performance of bivariate algorithms in inferring single links (at the microscale level). This provides further empirical evidence for the theoretical finding that bivariate TE—despite being a pairwise measure—does not depend solely on the directed link weight between a single pair of nodes, but on the larger network structure they are embedded in, via the mesoscopic network motifs (Novelli et al, 2020 ). In particular, the abundance of specific “clustered motifs” in modular structure increase the bivariate TE, making links within each group easier to detect but also increasing the false positive rate within modules.…”

“…At the microscale, the results concerning the bivariate TE can be explained in the light of the recent theoretical derivation of TE from network motifs for VAR dynamics (Novelli, Atay, Jost, & Lizier, 2020 ). For a fixed in-degree, the TE decreases with the rewiring probability, making it harder for candidate links to pass the statistical significance tests when only short time series are available.…”

“…An explanation is readily available in the simple case of VAR dynamics: The bivariate TE on spurious links is typically lower than the TE on real links, and it increases with the coupling strength. (Bivariate TE can be analytically derived from the network structure under the assumption of VAR dynamics, and spurious links are associated with larger motifs (e.g., longer chains), contributing to the TE at lower orders of magnitude (Novelli et al, 2020 ). Therefore, spurious links can only pass statistical significance tests when sufficiently long time series are available (in order for their weak TE values to be distinguished from noise); for the same reason, for shorter time series, spurious links can only be detected in the presence of strong enough coupling.…”

Synergistic information in a dynamical model implemented on the human structural connectome reveals spatially distinct associations with age

“…A weighted average of the synergy of each node, with properly fixed weights concentrated on the hubs of the strength, would also peak before criticality and thus would constitute a global precursor of the transition. Another interesting remark to be made is that, as noticed for example in Novelli, Atay, Jost and Lizier (2019), the pairwise transfer entropy from a source to a target node in a network does not depend solely on the local source-target link weight, but on the wider network structure that the link is embedded in. Deeply connected is the fact that the information flow in networks obeys the law of diminishing marginal returns; see Marinazzo, Wu, Pellicoro, Angelini, and Stramaglia (2012).…”

Maximized redundant and synergistic information transfers predict the rise in the output gene expression noise in a generic class of coherent type-1 feed-forward loop networks

“…II A. To establish the required analytical framework in discrete-time domain, we apply a stationary multivariate vector autoregressive (MVAR) process [31–33] as deliberated in the Appendix which also contains the details of the stochastic simulation method. The information-theoretic measures will be formally introduced in Sec.…”