A Systematic Exploration of Reservoir Computing for Forecasting Complex Spatiotemporal Dynamics

Abstract

A reservoir computer (RC) is a type of simplified recurrent neural network architecture that has demonstrated success in the prediction of spatiotemporally chaotic dynamical systems. A further advantage of RC is that it reproduces intrinsic dynamical quantities essential for its incorporation into numerical forecasting routines such as the ensemble Kalman filter-used in numerical weather prediction to compensate for sparse and noisy data. We explore here the architecture and design choices for a "best in class" RC for a number of characteristic dynamical systems, and then show the application of these choices in scaling up to larger models using localization. Our analysis points to the importance of large scale parameter optimization. We also note in particular the importance of including input bias in the RC design, which has a significant impact on the forecast skill of the trained RC model. In our tests, the the use of a nonlinear readout operator does not affect the forecast time or the stability of the forecast. The effects of the reservoir dimension, spinup time, amount of training data, normalization, noise, and the RC time step are also investigated. While we are not aware of a generally accepted best reported mean forecast time for different models in the literature, we report over a factor of 2 increase in the mean forecast time compared to the best performing RC model of [1] for the 40 dimensional spatiotemporally chaotic Lorenz 1996 dynamics, and we are able to accomplish this using a smaller reservoir size.

Learning Spatiotemporal Chaos Using Next-Generation Reservoir Computing

“…Recently, Platt et al [27] optimized the parallel RC architecture and obtained a mean prediction horizon twice as long as Vlachas et al using N in = 6 > N out = 2 with smaller RCs (each with d total = 720 nodes) and training data set (M = 40, 000). Our model obtains a slightly better performance in the mean prediction horizon with a computational complexity 5.6 × 10 3 shorter using 4 × 10 2 less training data than Platt.…”

Constraining chaos: Enforcing dynamical invariants in the training of reservoir computers

“…exceeds a certain value ε, in this case ε = 0.3 approximately in line with 6,45,48 . D is the system dimension, σ is the long term standard deviation of the time series and u f is the RC forecast.…”

“…When predicting the RC must first achieve synchronization with the input data 46 during a "spinup" phase before the prediction can begin. In 6,45,47 it is shown how to train the RC network through a two step training procedure that takes into account both the one step prediction accuracy as well as the long term forecast skill.…”

“…where u is the D×T matrix of input data, T is the number of time steps and r is the N ×T matrix of reservoir states 45 . We can add into the selection of these parameters prior knowledge of the global 6…”

“…An example forecast and distribution of predictions is shown in Fig. (2) where the conventionally trained 6,45 RC model successfully predicts the synoptic scale atmospheric dynamics for a number of months. Such significant predictive power on a low resolution QG model is an interesting result in and of itself, showcasing the ability of RNNs to resolve more realistic atmospheric dynamics.…”