Data-driven reduced-order modeling of spatiotemporal chaos with neural ordinary differential equations

Abstract

Dissipative partial differential equations that exhibit chaotic dynamics tend to evolve to attractors that exist on finite-dimensional manifolds. We present a data-driven reduced-order modeling method that capitalizes on this fact by finding a coordinate representation for this manifold and then a system of ordinary differential equations (ODEs) describing the dynamics in this coordinate system. The manifold coordinates are discovered using an undercomplete autoencoder—a neural network (NN) that reduces and then expands dimension. Then, the ODE, in these coordinates, is determined by a NN using the neural ODE framework. Both of these steps only require snapshots of data to learn a model, and the data can be widely and/or unevenly spaced. Time-derivative information is not needed. We apply this framework to the Kuramoto–Sivashinsky equation for domain sizes that exhibit chaotic dynamics with again estimated manifold dimensions ranging from 8 to 28. With this system, we find that dimension reduction improves performance relative to predictions in the ambient space, where artifacts arise. Then, with the low-dimensional model, we vary the training data spacing and find excellent short- and long-time statistical recreation of the true dynamics for widely spaced data (spacing of [Formula: see text] Lyapunov times). We end by comparing performance with various degrees of dimension reduction and find a “sweet spot” in terms of performance vs dimension.

Data-driven control of spatiotemporal chaos with reduced-order neural ODE-based models and reinforcement learning

“…This change of basis does not reduce dimension, and although NNs are universal function approximators, we found empirically in [48] that this change of basis results in a lower reconstruction error, indicating that this change of basis eases the training procedure. We identify the dimension of the finite dimensional manifold, dscriptM, by tracking the MSE performance of the autoencoders as we vary dh [27,48]. We comment that we use a standard autoencoder here as we and others [27,53] have found empirically that it performs well and yields a manifold representation that is conducive to forecasting tasks.…”

“…We identify the dimension of the finite dimensional manifold, dscriptM, by tracking the MSE performance of the autoencoders as we vary dh [27,48]. We comment that we use a standard autoencoder here as we and others [27,53] have found empirically that it performs well and yields a manifold representation that is conducive to forecasting tasks. Reference [53] found that variational autoencoders and convolutional autoencoders exhibited no clear advantage over standard feedforward autoencoders for the KSE system as well as the FitzHugh–Nagumo equation.…”

“…where h ∈ R d h and d h d. Before describing our specific approach, we briefly review other methods for developing dynamic models from data. A more detailed discussion is given in [27].…”

“…Alternately, one could simply represent f with a neural network [45]. If one does not have time-derivative data or estimates, then the neural ODE approach [27,46] can be used.…”

“…The dynamics of formally infinite-dimensional partial differential equations describing dissipative flow systems are known to collapse onto a finite-dimensional invariant manifold [47]—a so-called inertial manifold. In the past studies, it was shown that one can capture a high-dimensional system’s dynamics on this lower-dimensional manifold by using a combination of autoencoders, to identify the manifold coordinates, and neural ODEs (NODEs) [46], to model the dynamics on the manifold in discrete time [48] or continuous time [27]. Our aim is to adapt this data-driven, low-dimensional model framework as a data-efficient surrogate training grounds for model-based RL.…”