Constraining chaos: Enforcing dynamical invariants in the training of reservoir computers

Platt, Jason A.; Penny, Stephen G.; Smith, Timothy A.; Chen, Tse-Chun; Abarbanel, Henry D. I.

doi:10.1063/5.0156999

Cited by 2 publications

(1 citation statement)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…More recently, Platt et al. (2023) showed that constraining these macro‐scale parameters using global invariant properties of the underlying system leads the optimization algorithm to select parameters that generalize well to unseen test data. In that work, the authors were successful in using the largest positive Lyapunov exponent, and to a lesser extent the fractal dimension of the system.…”

Section: Echo State Network Prediction Skillmentioning

confidence: 99%

Temporal Subsampling Diminishes Small Spatial Scales in Recurrent Neural Network Emulators of Geophysical Turbulence

Smith,

Penny,

Platt

et al. 2023

J Adv Model Earth Syst

Self Cite

View full text Add to dashboard Cite

The immense computational cost of traditional numerical weather and climate models has sparked the development of machine learning (ML) based emulators. Because ML methods benefit from long records of training data, it is common to use data sets that are temporally subsampled relative to the time steps required for the numerical integration of differential equations. Here, we investigate how this often overlooked processing step affects the quality of an emulator's predictions. We implement two ML architectures from a class of methods called reservoir computing: (a) a form of Nonlinear Vector Autoregression (NVAR), and (b) an Echo State Network (ESN). Despite their simplicity, it is well documented that these architectures excel at predicting low dimensional chaotic dynamics. We are therefore motivated to test these architectures in an idealized setting of predicting high dimensional geophysical turbulence as represented by Surface Quasi‐Geostrophic dynamics. In all cases, subsampling the training data consistently leads to an increased bias at small spatial scales that resembles numerical diffusion. Interestingly, the NVAR architecture becomes unstable when the temporal resolution is increased, indicating that the polynomial based interactions are insufficient at capturing the detailed nonlinearities of the turbulent flow. The ESN architecture is found to be more robust, suggesting a benefit to the more expensive but more general structure. Spectral errors are reduced by including a penalty on the kinetic energy density spectrum during training, although the subsampling related errors persist. Future work is warranted to understand how the temporal resolution of training data affects other ML architectures.

show abstract

Section: Echo State Network Prediction Skillmentioning

confidence: 99%

Temporal Subsampling Diminishes Small Spatial Scales in Recurrent Neural Network Emulators of Geophysical Turbulence

Smith,

Penny,

Platt

et al. 2023

J Adv Model Earth Syst

Self Cite

View full text Add to dashboard Cite

show abstract

A hypothesis on ergodicity and the signal‐to‐noise paradox

Brener

2024

Atmospheric Science Letters

View full text Add to dashboard Cite

This letter raises the possibility that ergodicity concerns might have some bearing on the signal‐to‐noise paradox. This is explored by applying the ergodic theorem to the theory behind ensemble weather forecasting and the ensemble mean. Using the ensemble mean as our best forecast of observations amounts to interpreting it as the most likely phase‐space trajectory, which relies on the ergodic theorem. This can fail for ensemble forecasting systems if members are not perfectly exchangeable with each other, the averaging window is too short and/or there are too few members. We argue these failures can occur in cases such as the winter North Atlantic Oscillation (NAO) forecasts due to intransitivity or regime behaviour for regions such as the North Atlantic and Arctic. This behaviour, where different ensemble members may become stuck in different relatively persistent flow states (intransitivity) or multi‐modality (regime behaviour), can in certain situations break the ergodic theorem. The problem of non‐ergodic systems and models in the case of weather forecasting is discussed, as are potential mitigation methods and metrics for ergodicity in ensemble systems.

show abstract

Constraining chaos: Enforcing dynamical invariants in the training of reservoir computers

Cited by 2 publications

References 60 publications

Temporal Subsampling Diminishes Small Spatial Scales in Recurrent Neural Network Emulators of Geophysical Turbulence

Temporal Subsampling Diminishes Small Spatial Scales in Recurrent Neural Network Emulators of Geophysical Turbulence

A hypothesis on ergodicity and the signal‐to‐noise paradox

Contact Info

Product

Resources

About