Network structure effects in reservoir computers

Carroll, Thomas L.; Pecora, Louis M.

doi:10.1063/1.5097686

Cited by 67 publications

(50 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…First attempts in this direction for a reservoir with unweighted edges have recently been reported in Carroll and Pecora. 33 Current and future work is dedicated to the investigation of these questions -not the least because the answers to them will shed new light on the complexity of the underlying dynamical system.…”

Section: Discussionmentioning

confidence: 99%

Good and bad predictions: Assessing and improving the replication of chaotic attractors by means of reservoir computing

Haluszczynski

Räth

2019

Chaos: An Interdisciplinary Journal of Nonlinear Science

View full text Add to dashboard Cite

The prediction of complex nonlinear dynamical systems with the help of machine learning techniques has become increasingly popular. In particular, reservoir computing turned out to be a very promising approach especially for the reproduction of the long-term properties of a nonlinear system. Yet, a thorough statistical analysis of the forecast results is missing. Using the Lorenz and Rössler system we statistically analyze the quality of prediction for different parametrizations -both the exact short-term prediction as well as the reproduction of the long-term properties (the "climate") of the system as estimated by the correlation dimension and largest Lyapunov exponent. We find that both short and longterm predictions vary significantly among the realizations. Thus special care must be taken in selecting the good predictions as predictions which deliver better short-term prediction also tend to better resemble the long-term climate of the system. Instead of only using purely random Erdös-Renyi networks we also investigate the benefit of alternative network topologies such as small world or scale-free networks and show which effect they have on the prediction quality. Our results suggest that the overall performance with respect to the reproduction of the climate of both the Lorenz and Rössler system is worst for scale-free networks. For the Lorenz system there seems to be a slight benefit of using small world networks while for the Rössler system small world and Erdös-Renyi networks performed equivalently well. In general the observation is that reservoir computing works for all network topologies investigated here.

show abstract

Section: Discussionmentioning

confidence: 99%

Good and bad predictions: Assessing and improving the replication of chaotic attractors by means of reservoir computing

Haluszczynski

Räth

2019

Chaos: An Interdisciplinary Journal of Nonlinear Science

View full text Add to dashboard Cite

show abstract

“…Rather, in this work the reservoir is being used to learn the nonlinear function between an input signal and a training signal. In [26] showed, this function approximation is similar to fitting a signal with a set of orthogonal basis functions; the higher the rank of this basis (the covariance rank in this paper), the better the fit. In addition, the reservoir must be synchronized to the input signal in the general sense.…”

Section: Edge Of Chaosmentioning

confidence: 88%

“…The maximum covariance rank is equal to the number of nodes, M = 100. In [26], higher covariance rank was associated with lower testing error.…”

Section: A Covariance Rankmentioning

confidence: 88%

See 1 more Smart Citation

Dimension of reservoir computers

Carroll

2020

Chaos: An Interdisciplinary Journal of Nonlinear Science

Self Cite

View full text Add to dashboard Cite

A reservoir computer is a complex dynamical system, often created by coupling nonlinear nodes in a network. The nodes are all driven by a common driving signal. In this work, three dimension estimation methods, false nearest neighbor, covariance and Kaplan-Yorke dimensions, are used to estimate the dimension of the reservoir dynamical system. It is shown that the signals in the reservoir system exist on a relatively low dimensional surface. Changing the spectral radius of the reservoir network can increase the fractal dimension of the reservoir signals, leading to an increase in testing error.A reservoir computer uses a complex dynamical system to perform computations. The reservoir is often created by coupling together a set of nonlinear nodes. Each node is driven by a common input signal. The time series responses from each node are then used to fit a training signal that is related to the input. The training can take place via a least squares fit, while, the connections between nodes are not altered during training, so training a reservoir computer is fast.Reservoir computers are described as "high dimensional" dynamical systems because they contain many signals, but the concept of dimension is rarely explored. A reservoir with M nodes defines an M dimensional space, but the actual signals in the reservoir may live on a lower dimensional surface. Two different dimension estimation methods are used to find the dimension of this surface. Counterintuitively, as the dimension of this surface increases, the fits to the training signal become worse. The increase in reservoir dimension can be explained by a well known effect in driven dynamical systems that causes signals in the driven system to have a higher fractal dimension than the driving signal. This increase in fractal dimension leads to worse performance for the reservoir computer.

show abstract

“…The readout or output layer is the only part where the weights are trained to produce a desired output which should be closer to the target data. Researchers have also devoted to find the optimal parameters of an ESN for accurate detection of target data [13,[19][20][21][22][23][24][25].…”

Section: Introductionmentioning

confidence: 99%

Predicting bursting in a complete graph of mixed population through reservoir computing

Saha

Mishra

Ghosh

et al. 2020

Phys. Rev. Research

View full text Add to dashboard Cite

We report our investigation and success story, to an extent, on the prediction of spiking and bursting dynamics in globally coupled networks, using echo state network/reservoir computing-based learning procedure. Two exemplary dynamical models, Josephson junctions and Hindmarsh-Rose neurons, are used to construct two separate networks and thereby illustrate the efficacy of our strategy. In the absence of coupling, the networks consist of mixed populations in which few nodes are oscillatory (self-sustained spiking) and the rest of the nodes maintain a quiescent state. When single-input data from one oscillatory node of a network (under stronger interactions between the nodes) is used for learning, the ESN is able to predict the key dynamical features (spiking and bursting) of the other nodes. In comparison, the machine performs with improved predictions if it is fed with two inputs: one from the oscillatory population and another from an excitable population. The machine's leaking parameter plays a crucial role, which can be tuned appropriately to enhance prediction. Furthermore, a cluster synchronization in the mixed population is confirmed from the machine-generated output signals. Our work is expected to be useful as a burst predictor.

show abstract

Network structure effects in reservoir computers

Cited by 67 publications

References 27 publications

Good and bad predictions: Assessing and improving the replication of chaotic attractors by means of reservoir computing

Good and bad predictions: Assessing and improving the replication of chaotic attractors by means of reservoir computing

Dimension of reservoir computers

Predicting bursting in a complete graph of mixed population through reservoir computing

Contact Info

Product

Resources

About