Spatiotemporal signal propagation in complex networks

Hens, Chittaranjan; Harush, Uzi; Haber, Simi; Cohen, Reuven; Barzel, Baruch

doi:10.1038/s41567-018-0409-0

Cited by 174 publications

(150 citation statements)

References 41 publications

Supporting

Mentioning

146

Contrasting

Order By: Relevance

“…Through the above analysis, we find that, similarly with complex networks, there is an interesting spatiotemporal information representation and propagation in DNNs, where information capability becomes more and more abundant by increasing spatial representation and the evolution towards the edge of chaos as depth grows [29]. The spatiotemporal structures can also be modeled by feature selection.…”

Section: Dynamics Of a Vanilla Deep Neural Networkmentioning

confidence: 96%

The expressivity and training of deep neural networks: Toward the edge of chaos?

Zhang

Li²,

Shen

et al. 2020

Neurocomputing

View full text Add to dashboard Cite

Expressivity is one of the most significant issues in assessing neural networks. In this paper, we provide a quantitative analysis of the expressivity for the deep neural network (DNN) from its dynamic model, where the Hilbert space is employed to analyze the convergence and criticality. We study the feature mapping of several widely used activation functions obtained by Hermite polynomials, and find sharp declines or even saddle points in the feature space, which stagnate the information transfer in DNNs. We then present a new activation function design based on the Hermite polynomials for better utilization of spatial representation. Moreover, we analyze the information transfer of DNNs, emphasizing the convergence problem caused by the mismatch between input and topological structure. We also study the effects of input perturbations and regularization operators on critical expressivity. Our theoretical analysis reveals that DNNs use spatial domains for information representation and evolve to the edge of chaos as depth increases. In actual training, whether a particular network can ultimately arrive the edge of chaos depends on its ability to overcome convergence and pass information to the required network depth. Finally, we demonstrate the empirical performance of the proposed hypothesis via multivariate time series prediction and image classification examples.

show abstract

Section: Dynamics Of a Vanilla Deep Neural Networkmentioning

confidence: 96%

The expressivity and training of deep neural networks: Toward the edge of chaos?

Zhang

Li²,

Shen

et al. 2020

Neurocomputing

View full text Add to dashboard Cite

show abstract

“…This is reflected in parameter uncertainty. As such, data monitoring of the network [12], [13] is essential for both scientific study and maintaining operational capacity. Therefore, the purpose of this paper is to study how to sample and recover the signals in the dynamic network from the combining time and graph -domains.…”

Section: System Modelmentioning

confidence: 99%

“…This is reflected in parameter uncertainty. As such, data monitoring of the network [12], [13] is essential for both scientific study and maintaining operational capacity. We propose to use the deterministic complex system framework to identify the optimal Zhuangkun Wei, and Weisi Guo are with the School of Engineering, the University of Warwick, West Midlands, CV47AL, UK.…”

Section: I Introductionmentioning

confidence: 99%

Optimal Sampling for Dynamic Complex Networks With Graph-Bandlimited Initialization

Wei

Guo

2019

IEEE Access

View full text Add to dashboard Cite

Many engineering, social, and biological complex systems consist of dynamical elements connected via a large-scale network. Monitoring the network's dynamics is essential for a variety of maintenance and scientific purposes. Whilst we understand how to optimally sample a single dynamic element or a nondynamic graph, we do not possess a theory on how to optimally sample networked dynamical elements. Here, we study nonlinear dynamic graph signals on a fixed complex network. We define the necessary conditions for optimal sampling in the combining timeand graph-domain to fully recover the networked dynamics. We firstly interpret the networked dynamics into a linearized matrix. Then, we prove that the dynamic signals can be sampled and fully recovered if the networked dynamics is stable and their initialization is bandlimited in the graph spectral domain. This new theory directly maps optimal sampling locations and rates to the graph properties and governing nonlinear dynamics. This can inform the optimal placement of experimental probes and sensors on dynamical networks as well as inform the design of each sensor's optimal sampling rate. We motivate the reader with two examples of recovering the networked dynamics for: social population growth and networked protein biochemical interactions with both bandlimited and arbitrary initialization.

show abstract

“…In fact, it has been recently shown that the relation between structure and dynamics of information flow can be unraveled by analyzing how perturbations propagate through the system [22]. The complex interplay between structure and dynamics of information propagation has been also mapped to its latent geometry to better understand network-driven contagion phenomena [23] and, more recently, it has been shown how signal propagation is able to capture the role of network connectivity in propagating local information, thus linking the topology to the observed spatio-temporal spread of perturbative signals across it [24]. However, the same task is even more challenging when the system of interest can be described in terms of a multilayer network [25][26][27][28][29][30][31].…”

Section: Introductionmentioning

confidence: 99%

Enhancing transport properties in interconnected systems without altering their structure

Ghavasieh

Domenico

2020

Phys. Rev. Research

View full text Add to dashboard Cite

Units of complex systems -such as neurons in the brain or individuals in societies -must communicate efficiently to function properly: e.g., allowing electrochemical signals to travel quickly among functionally connected neuronal areas in the human brain, or allowing for fast navigation of humans and goods in complex transportation landscapes. The coexistence of different types of relationships among the units, entailing a multilayer represention in which types are considered as networks encoded by layer s, plays an important role in the quality of information exchange among them. While altering the structure of such systems -e.g., by physically adding (or removing) units, connections or layers -might be costly, coupling the dynamics of subset(s) of layers in a way that reduces the number of redundant diffusion pathways across the multilayer system, can potentially accelerate the overall information flow. To this aim, we introduce a framework for functional reducibility which allow us to enhance transport phenomena in multilayer systems by coupling layers together with respect to dynamics rather than structure. Mathematically, the optimal configuration is obtained by maximizing the deviation of system's entropy from the limit of free and non-interacting layers. Our results provide a transparent procedure to reduce diffusion time and optimize non-compact search processes in empirical multilayer systems, without the cost of altering the underlying structure.

show abstract

Spatiotemporal signal propagation in complex networks

Cited by 174 publications

References 41 publications

The expressivity and training of deep neural networks: Toward the edge of chaos?

The expressivity and training of deep neural networks: Toward the edge of chaos?

Optimal Sampling for Dynamic Complex Networks With Graph-Bandlimited Initialization

Enhancing transport properties in interconnected systems without altering their structure

Contact Info

Product

Resources

About