Data-driven predictions of a multiscale Lorenz 96 chaotic system using machine-learning methods: reservoir computing, artificial neural network, and long short-term memory network

Chattopadhyay, Ashesh; Hassanzadeh, Pedram; Subramanian, Devika

doi:10.5194/npg-27-373-2020

Cited by 161 publications

(99 citation statements)

References 74 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…ML methods (in this case, recurrent neural networks) have also shown themselves capable of reproducing a time series from canonical chaotic systems with predictability beyond what dynamical systems theory would suggest, e.g. [ 55 ] (which indeed explicitly makes a claim to be ‘model-free’), [ 56 ]. Does this mean we have come full circle on the von Neumann revolution, and return to forecasting from pattern recognition rather than physics?…”

Section: Learning Physics From Datamentioning

confidence: 99%

Climbing down Charney’s ladder: machine learning and the post-Dennard era of computational climate science

Balaji

2021

Phil. Trans. R. Soc. A.

View full text Add to dashboard Cite

The advent of digital computing in the 1950s sparked a revolution in the science of weather and climate. Meteorology, long based on extrapolating patterns in space and time, gave way to computational methods in a decade of advances in numerical weather forecasting. Those same methods also gave rise to computational climate science, studying the behaviour of those same numerical equations over intervals much longer than weather events, and changes in external boundary conditions. Several subsequent decades of exponential growth in computational power have brought us to the present day, where models ever grow in resolution and complexity, capable of mastery of many small-scale phenomena with global repercussions, and ever more intricate feedbacks in the Earth system. The current juncture in computing, seven decades later, heralds an end to what is called Dennard scaling, the physics behind ever smaller computational units and ever faster arithmetic. This is prompting a fundamental change in our approach to the simulation of weather and climate, potentially as revolutionary as that wrought by John von Neumann in the 1950s. One approach could return us to an earlier era of pattern recognition and extrapolation, this time aided by computational power. Another approach could lead us to insights that continue to be expressed in mathematical equations. In either approach, or any synthesis of those, it is clearly no longer the steady march of the last few decades, continuing to add detail to ever more elaborate models. In this prospectus, we attempt to show the outlines of how this may unfold in the coming decades, a new harnessing of physical knowledge, computation and data. This article is part of the theme issue ‘Machine learning for weather and climate modelling’.

show abstract

Section: Learning Physics From Datamentioning

confidence: 99%

Climbing down Charney’s ladder: machine learning and the post-Dennard era of computational climate science

Balaji

2021

Phil. Trans. R. Soc. A.

View full text Add to dashboard Cite

show abstract

“…RNNs are networks in which, unlike ANNs, information moves forward and backward between the input, deeper layers, and output. RNNs account for memory between sequential data points and are often used for time series forecasting (Chattopadhyay, Hassanzadeh, & Subramanian, 2020; Goodfellow et al, 2016). We use a gated recurrent unit (GRU) as the RNN (see Appendix B for details).…”

Section: The Numerical and Data‐driven (Super‐)parameterized Modelsmentioning

confidence: 99%

“…Although the focus of this paper is on subgrid‐scale modeling, we have also investigated the performance of a fully DD model. In this model, following Chattopadhyay, Hassanzadeh, & Subramanian (2020), an RNN is trained on X , which is then used to predict the spatiotemporal evolution of X from an initial condition. The RNN we use here is a GRU (see Appendix C), trained on 10 6 sequential values of X ( t ) sampled at every 10Δ t from the DNS data to mimic the time step used in all other low‐resolution models.…”

Section: The Numerical and Data‐driven (Super‐)parameterized Modelsmentioning

confidence: 99%

“…Here P is a physics‐based or semi‐empirical function (Balaji, 2020; Chattopadhyay, Hassanzadeh, & Subramanian, 2020; Hourdin et al, 2017; Jeevanjee et al, 2017; Palmer, 2019; Rasp, 2019; Rasp et al, 2018; Schneider et al, 2017). Despite much effort, some of these often‐heuristic parameterization schemes (e.g., for clouds or gravity waves) have major shortcomings that result in persistent biases and large uncertainties, which degrade weather forecasts and climate change projections (Alexander et al, 2010; Bony et al, 2015; Polichtchouk et al, 2018; Schneider et al, 2017; Sigmond & Scinocca, 2010; Stevens & Bony, 2013).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Data‐Driven Super‐Parameterization Using Deep Learning: Experimentation With Multiscale Lorenz 96 Systems and Transfer Learning

Chattopadhyay

Subel

Hassanzadeh

2020

J Adv Model Earth Syst

Self Cite

View full text Add to dashboard Cite

To make weather and climate models computationally affordable, small-scale processes are usually represented in terms of the large-scale, explicitly resolved processes using physics-based/ semi-empirical parameterization schemes. Another approach, computationally more demanding but often more accurate, is super-parameterization (SP). SP involves integrating the equations of small-scale processes on high-resolution grids embedded within the low-resolution grid of large-scale processes. Recently, studies have used machine learning (ML) to develop data-driven parameterization (DD-P) schemes. Here, we propose a new approach, data-driven SP (DD-SP), in which the equations of the small-scale processes are integrated data-drivenly (thus inexpensively) using ML methods such as recurrent neural networks. Employing multiscale Lorenz 96 systems as the testbed, we compare the cost and accuracy (in terms of both short-term prediction and long-term statistics) of parameterized low-resolution (PLR) SP, DD-P, and DD-SP models. We show that with the same computational cost, DD-SP substantially outperforms PLR and is more accurate than DD-P, particularly when scale separation is lacking. DD-SP is much cheaper than SP, yet its accuracy is the same in reproducing long-term statistics (climate prediction) and often comparable in short-term forecasting (weather prediction). We also investigate generalization: when models trained on data from one system are applied to a more chaotic system, we find that models often do not generalize, particularly when short-term prediction accuracies are examined. However, we show that transfer learning, which involves retraining the data-driven model with a small amount of data from the new system, significantly improves generalization. Potential applications of DD-SP and transfer learning in climate/weather modeling are discussed. Plain Language Summary The weather/climate system involves intertwined physical processes acting on scales from centimeters (or even smaller) to tens of thousands of kilometers. Most weather/climate models used in practice include parameterization schemes that relate small-scale processes, which are not explicitly resolved (due to coarse spatiotemporal resolution), to large-scale processes that are resolved. Recently, studies have explored using machine learning for data-driven parameterization (DD-P) of small-scale (subgrid) processes. Here, we first introduce a novel way to leverage recent advances in deep learning to improve the modeling of subgrid processes. In this approach, called data-driven super-parameterization (DD-SP), deep learning is used for fast, data-driven integration of equations of small-scale processes, while other equations are integrated using conventional numerical methods. Employing a relatively simple chaotic system, we show the advantages of DD-SP over DD-P and conventional parameterizations. Second, we examine how these data-driven models generalize (extrapolate) from one system to other (e.g., more chaotic) systems. We demonstrate that these mod...

show abstract

“…The L96 model has rapidly gained relevance among geoscientists, physicists, and applied mathematicians, as it has become a benchmark testbed for parametrisations [40][41][42][43][44][45][46], for studying extreme events [47][48][49][50], for developing data assimilation schemes [51][52][53][54], for developing ensemble forecasting techniques [55][56][57], for studying the properties of Lyapunov exponents and covariant Lyapunov vectors [58][59][60][61], for developing and testing ideas in nonequilibrium statistical mechanics [62][63][64][65][66], and for investigating bifurcations [67][68][69][70][71][72][73]. By looking at these references, the reader can find a very thorough analysis of the properties of the L96 model.…”

Section: The Lorenz '96 Modelmentioning

confidence: 99%

Mechanics and thermodynamics of a new minimal model of the atmosphere

Vissio

Lucarini

2020

Eur. Phys. J. Plus

View full text Add to dashboard Cite

The understanding of the fundamental properties of the climate system has long benefitted from the use of simple numerical models able to parsimoniously represent the essential ingredients of its processes. Here, we introduce a new model for the atmosphere that is constructed by supplementing the now-classic Lorenz ’96 one-dimensional lattice model with temperature-like variables. The model features an energy cycle that allows for energy to be converted between the kinetic form and the potential form and for introducing a notion of efficiency. The model’s evolution is controlled by two contributions—a quasi-symplectic and a gradient one, which resemble (yet not conforming to) a metriplectic structure. After investigating the linear stability of the symmetric fixed point, we perform a systematic parametric investigation that allows us to define regions in the parameters space where at steady-state stationary, quasi-periodic, and chaotic motions are realised, and study how the terms responsible for defining the energy budget of the system depend on the external forcing injecting energy in the kinetic and in the potential energy reservoirs. Finally, we find preliminary evidence that the model features extensive chaos. We also introduce a more complex version of the model that is able to accommodate for multiscale dynamics and that features an energy cycle that more closely mimics the one of the Earth’s atmosphere.

show abstract

Data-driven predictions of a multiscale Lorenz 96 chaotic system using machine-learning methods: reservoir computing, artificial neural network, and long short-term memory network

Cited by 161 publications

References 74 publications

Climbing down Charney’s ladder: machine learning and the post-Dennard era of computational climate science

Climbing down Charney’s ladder: machine learning and the post-Dennard era of computational climate science

Data‐Driven Super‐Parameterization Using Deep Learning: Experimentation With Multiscale Lorenz 96 Systems and Transfer Learning

Mechanics and thermodynamics of a new minimal model of the atmosphere

Contact Info

Product

Resources

About