Synthesis of recurrent neural networks for dynamical system simulation

Abstract

We review several of the most widely used techniques for training recurrent neural networks to approximate dynamical systems, then describe a novel algorithm for this task. The algorithm is based on an earlier theoretical result that guarantees the quality of the network approximation. We show that a feedforward neural network can be trained on the vector-field representation of a given dynamical system using backpropagation, then recast it as a recurrent network that replicates the original system's dynamics. After detailing this algorithm and its relation to earlier approaches, we present numerical examples that demonstrate its capabilities. One of the distinguishing features of our approach is that both the original dynamical systems and the recurrent networks that simulate them operate in continuous time.

Data-Driven Discovery of Closure Models

“…These include subjective visual inspection [21] or measures for the attractor [2] such as maximum Lyapunov exponent [17], correlation dimension and other time averaged characteristics [28]. The first approach, although perhaps the most widely used [30][39] [49], can sometimes be misleading [15].…”

Nonlinear Autoregressive Network with the Use of a Moving Average Method for Forecasting Typhoon Tracks

“…Relatively, the time factor is explicit represented in the dynamic neural network model, by using feedback loop to cause time delays. Further to mention that the dynamic neural network is not only treat nonlinear multivariate behaviour, but also include learning of time-dependent behaviour such as various transient phenomena and delay effects [21][22][23][24][25]. Nevertheless, both neural network models can be applied to analyse the present research topic in typhoon path prediction problem.…”

Predicting dynamical system evolution with residual neural networks

“…One typical problem is to reconstruct solution of an ODE (ordinary differential equation) systeṁ x = F (x) by learning the right hand side function F (x) with a suitable neural network. In [1] promising results for this problem are reported with a shallow neural network applied to learn F (x) and then transformed into a recurrent neural network (RNN).…”

“…However, in real life applications the right hand side function F (x) may not be known explicitly and the ODE system may only be described by sampled data points. In this case the approaches of [1,2] are not applicable. That is where machine learning, as a means of building models from the data, might be useful.…”