Machine Learning for Stochastic Parameterization: Generative Adversarial Networks in the Lorenz '96 Model

Gagne, David John; Christensen, Hilary; Subramanian, Aneesh; Monahan, Adam H.

doi:10.1029/2019ms001896

Cited by 130 publications

(123 citation statements)

References 88 publications

(120 reference statements)

Supporting

Mentioning

122

Contrasting

Order By: Relevance

“…Furthermore, for some poorly understood processes for which observational data are available (e.g., clouds), data-driven surrogate models built using such data might potentially outperform physics-based surrogate models [22,30]. Recent studies have shown promising results in using AI to build data-driven parameterizations for modeling of some atmospheric and oceanic processes [33,34,35,36,37,38,39,40]. In the turbulence and dynamical systems communities, similarly encouraging outcomes have been reported [41,42,43,44,45,46,26,47,48,49] .…”

Section: Introductionmentioning

confidence: 90%

See 1 more Smart Citation

Data-driven prediction of a multi-scale Lorenz 96 chaotic system using deep learning methods: Reservoir computing, ANN, and RNN-LSTM

Chattopadhyay¹,

Hassanzadeh²,

Subramanian³

2019

Preprint

View full text Add to dashboard Cite

In this paper, the performance of three deep learning methods for predicting short-term evolution and for reproducing the long-term statistics of a multi-scale spatio-temporal Lorenz 96 system is examined. The methods are: echo state network (a type of reservoir computing, RC-ESN), deep feed-forward artificial neural network (ANN), and recurrent neural network with long short-term memory (RNN-LSTM). This Lorenz 96 system has three tiers of nonlinearly interacting variables representing slow/largescale (X), intermediate (Y ), and fast/small-scale (Z) processes. For training or testing, only X is available; Y and Z are never known or used. We show that RC-ESN substantially outperforms ANN and RNN-LSTM for short-term prediction, e.g., accurately forecasting the chaotic trajectories for hundreds of numerical solver's time steps, equivalent to several Lyapunov timescales. The RNN-LSTM and ANN show some prediction skills as well; RNN-LSTM bests ANN. Furthermore, even after losing the trajectory, data predicted by RC-ESN and RNN-LSTM have probability density functions (PDFs) that closely match the true PDF, even at the tails. The PDF of the data predicted using ANN, however, deviates from the true PDF. Implications, caveats, and applications to data-driven and data-assisted surrogate modeling of complex nonlinear dynamical systems such as weather/climate are discussed.

show abstract

Section: Introductionmentioning

confidence: 90%

“…As seen in Figure 5(A), RC-ESN accurately predicts the time series for over 2.3 MTU, which is equivalent to 460∆t and over 10. 35 Lyapunov timescales. Closer examination shows that the RC-RSN prediction follows the true trajectory well even up to ≈ 4 MTU.…”

Section: Short-term Prediction: Comparison Of the Rc-esn Ann And Rnmentioning

confidence: 99%

Data-driven prediction of a multi-scale Lorenz 96 chaotic system using deep learning methods: Reservoir computing, ANN, and RNN-LSTM

Chattopadhyay¹,

Hassanzadeh²,

Subramanian³

2019

Preprint

View full text Add to dashboard Cite

show abstract

“…In our study we trained a deep neural network to emulate the CRM tendencies. The offline validation scores were very encouraging (Gentine et al, 2018) even though the deterministic ML parameterization was unable to reproduce the variability in the boundary layer. When we subse-quently implemented the ML parameterization in the climate model and ran it prognostically (online), we managed to engineer a stable model that produced results close to the original SP-GCM.…”

Section: Rasp Et Al (2018) -Super-parameterization With a Neural Netmentioning

confidence: 98%

Coupled online learning as a way to tackle instabilities and biases in neural network parameterizations: general algorithms and Lorenz 96 case study (v1.0)

Rasp

2020

Geosci. Model Dev.

View full text Add to dashboard Cite

Abstract. Over the last couple of years, machine learning parameterizations have emerged as a potential way to improve the representation of subgrid processes in Earth system models (ESMs). So far, all studies were based on the same three-step approach: first a training dataset was created from a high-resolution simulation, then a machine learning algorithm was fitted to this dataset, before the trained algorithm was implemented in the ESM. The resulting online simulations were frequently plagued by instabilities and biases. Here, coupled online learning is proposed as a way to combat these issues. Coupled learning can be seen as a second training stage in which the pretrained machine learning parameterization, specifically a neural network, is run in parallel with a high-resolution simulation. The high-resolution simulation is kept in sync with the neural network-driven ESM through constant nudging. This enables the neural network to learn from the tendencies that the high-resolution simulation would produce if it experienced the states the neural network creates. The concept is illustrated using the Lorenz 96 model, where coupled learning is able to recover the “true” parameterizations. Further, detailed algorithms for the implementation of coupled learning in 3D cloud-resolving models and the super parameterization framework are presented. Finally, outstanding challenges and issues not resolved by this approach are discussed.

show abstract

“…As a result, to make the simulations feasible, a few strategies have been developed, which mainly involve only solving for slow/large-scale variables and accounting for the fast/small-scale processes through surrogate models. In weather and climate models, for example, semiempirical physics-based parameterizations are often used to represent the effects of processes such as gravity waves and moist convection in the atmosphere or submesoscale eddies in the ocean (Stevens and Bony, 2013;Hourdin et al, 2017;Garcia et al, 2017;Jeevanjee et al, 2017;Schneider et al, 2017b;Chattopadhyay et al, 2020b, a). A more advanced approach is "super-parameterization" which, for example, involves solving the PDEs of moist convection on a high-resolution grid inside each grid point of large-scale atmospheric circulation for which the governing PDEs (the Navier-Stokes equations) are solved on a coarse grid (Khairoutdinov and Randall, 2001).…”

Section: Introductionmentioning

confidence: 99%

Data-driven predictions of a multiscale Lorenz 96 chaotic system using machine-learning methods: reservoir computing, artificial neural network, and long short-term memory network

Chattopadhyay

Hassanzadeh

Subramanian

2020

Nonlin. Processes Geophys.

161

View full text Add to dashboard Cite

Abstract. In this paper, the performance of three machine-learning methods for predicting short-term evolution and for reproducing the long-term statistics of a multiscale spatiotemporal Lorenz 96 system is examined. The methods are an echo state network (ESN, which is a type of reservoir computing; hereafter RC–ESN), a deep feed-forward artificial neural network (ANN), and a recurrent neural network (RNN) with long short-term memory (LSTM; hereafter RNN–LSTM). This Lorenz 96 system has three tiers of nonlinearly interacting variables representing slow/large-scale (X), intermediate (Y), and fast/small-scale (Z) processes. For training or testing, only X is available; Y and Z are never known or used. We show that RC–ESN substantially outperforms ANN and RNN–LSTM for short-term predictions, e.g., accurately forecasting the chaotic trajectories for hundreds of numerical solver's time steps equivalent to several Lyapunov timescales. The RNN–LSTM outperforms ANN, and both methods show some prediction skills too. Furthermore, even after losing the trajectory, data predicted by RC–ESN and RNN–LSTM have probability density functions (pdf's) that closely match the true pdf – even at the tails. The pdf of the data predicted using ANN, however, deviates from the true pdf. Implications, caveats, and applications to data-driven and data-assisted surrogate modeling of complex nonlinear dynamical systems, such as weather and climate, are discussed.

show abstract

Machine Learning for Stochastic Parameterization: Generative Adversarial Networks in the Lorenz '96 Model

Cited by 130 publications

References 88 publications

Data-driven prediction of a multi-scale Lorenz 96 chaotic system using deep learning methods: Reservoir computing, ANN, and RNN-LSTM

Data-driven prediction of a multi-scale Lorenz 96 chaotic system using deep learning methods: Reservoir computing, ANN, and RNN-LSTM

Coupled online learning as a way to tackle instabilities and biases in neural network parameterizations: general algorithms and Lorenz 96 case study (v1.0)

Data-driven predictions of a multiscale Lorenz 96 chaotic system using machine-learning methods: reservoir computing, artificial neural network, and long short-term memory network

Contact Info

Product

Resources

About