Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks

Abstract

We introduce a data-driven forecasting method for high-dimensional chaotic systems using long short-term memory (LSTM) recurrent neural networks. The proposed LSTM neural networks perform inference of high-dimensional dynamical systems in their reduced order space and are shown to be an effective set of nonlinear approximators of their attractor. We demonstrate the forecasting performance of the LSTM and compare it with Gaussian processes (GPs) in time series obtained from the Lorenz 96 system, the Kuramoto-Sivashinsky equation and a prototype climate model. The LSTM networks outperform the GPs in short-term forecasting accuracy in all applications considered. A hybrid architecture, extending the LSTM with a mean stochastic model (MSM-LSTM), is proposed to ensure convergence to the invariant measure. This novel hybrid method is fully data-driven and extends the forecasting capabilities of LSTM networks.

Toward Data‐Driven Weather and Climate Forecasting: Approximating a Simple General Circulation Model With Deep Learning

“…They use convolutional kernels to process information from one layer to the next, and after each convolution layer there is a nonlinear transformation. A big advantage of CNNs compared to other machine learning methods is that there is no need for prior dimensionality reduction, as in, for example, Vlachas et al (2018). It is common to represent atmospheric data on regular grids.…”

“…For the same reasons, the simplest possible model setup was chosen, with no seasonal cycle (eternal Northern Hemispheric winter), no orography, a horizontal resolution of T21(∼625 km, 32 × 64 grid points when projected on a regular latlon grid), 10 vertical levels, no diurnal cycle, no ocean, and a time step of 45 min. Still, the model is more complex and closer to the real atmosphere than, for example, the barotropic model which Vlachas et al (2018) approximated with a neural network. For the final data, the model's four state variables (horizontal and meridional wind, temperature, and geopotential height) are interpolated from the 10 model levels to 10 pressure levels.…”

“…This makes it interesting to use CNNs in connection with atmospheric models. A big advantage of CNNs compared to other machine learning methods is that there is no need for prior dimensionality reduction, as in, for example, Vlachas et al (2018). CNNs have already been used in postprocessing of GCM output in order to detect extreme events (Liu et al, 2016;Racah et al, 2016).…”

“…A lot of progress has been reported on this topic for very idealized models like the Lorenz63 and Lorenz96 models (Lorenz, 1963(Lorenz, , 1996 and simple barotropic climate models (e.g., Karunasinghe & Liong, 2006;Krasnopolsky & Fox-Rabinovitz, 2006;Krasnopolsky et al, 2013;Vlachas et al, 2018), and recently also some progress for the prediction of geopotential height in a simplified weather forecasting setting (Dueben & Bauer, 2018). A lot of progress has been reported on this topic for very idealized models like the Lorenz63 and Lorenz96 models (Lorenz, 1963(Lorenz, , 1996 and simple barotropic climate models (e.g., Karunasinghe & Liong, 2006;Krasnopolsky & Fox-Rabinovitz, 2006;Krasnopolsky et al, 2013;Vlachas et al, 2018), and recently also some progress for the prediction of geopotential height in a simplified weather forecasting setting (Dueben & Bauer, 2018).…”

“…Here we use deep learning not to extract information from a climate model, or to combine different models, but to directly emulate the complete physics and dynamics of a GCM, generating a neural network that takes as its input the complete model state of the GCM and then predicts the next model state. A lot of progress has been reported on this topic for very idealized models like the Lorenz63 and Lorenz96 models (Lorenz, 1963(Lorenz, , 1996 and simple barotropic climate models (e.g., Karunasinghe & Liong, 2006;Krasnopolsky & Fox-Rabinovitz, 2006;Krasnopolsky et al, 2013;Vlachas et al, 2018), and recently also some progress for the prediction of geopotential height in a simplified weather forecasting setting (Dueben & Bauer, 2018). However, to the author's best knowledge, using machine learning methods to map the full state of a GCM to its full state several time steps later-thus learning the complete model dynamics-has not been done before.…”