Climbing down Charney’s ladder: machine learning and the post-Dennard era of computational climate science

Abstract

The advent of digital computing in the 1950s sparked a revolution in the science of weather and climate. Meteorology, long based on extrapolating patterns in space and time, gave way to computational methods in a decade of advances in numerical weather forecasting. Those same methods also gave rise to computational climate science, studying the behaviour of those same numerical equations over intervals much longer than weather events, and changes in external boundary conditions. Several subsequent decades of exponential growth in computational power have brought us to the present day, where models ever grow in resolution and complexity, capable of mastery of many small-scale phenomena with global repercussions, and ever more intricate feedbacks in the Earth system. The current juncture in computing, seven decades later, heralds an end to what is called Dennard scaling, the physics behind ever smaller computational units and ever faster arithmetic. This is prompting a fundamental change in our approach to the simulation of weather and climate, potentially as revolutionary as that wrought by John von Neumann in the 1950s. One approach could return us to an earlier era of pattern recognition and extrapolation, this time aided by computational power. Another approach could lead us to insights that continue to be expressed in mathematical equations. In either approach, or any synthesis of those, it is clearly no longer the steady march of the last few decades, continuing to add detail to ever more elaborate models. In this prospectus, we attempt to show the outlines of how this may unfold in the coming decades, a new harnessing of physical knowledge, computation and data. This article is part of the theme issue ‘Machine learning for weather and climate modelling’.

Revealing the Impact of Global Heating on North Atlantic Circulation Using Transparent Machine Learning

“…Using supervised ML, being able to explain the source of predictive skill and move beyond a “black box” approach, to create transparency, is often nontrivial. This difficulty should not detract from the importance of transparent ML applications, as leveraging the combination of domain knowledge and emerging ML techniques such as AFA could be of pivotal importance for applications within the physical sciences (Balaji, 2020; Irrgang et al., 2021; McGovern et al., 2019; Sonnewald et al., 2021; Toms et al., 2020). When used as a “black box”, a NN will be trained to make desired prediction, and while it can be skillful in making these predictions, it could have skill rooted in chance more than physics.…”

“…To be truly appropriate for application to the physical sciences, the source of skill from ML should be transparent. At the root of this need is a necessity that the ML is based on something physical and not random chance (Balaji, 2020;Irrgang et al, 2021;Sonnewald et al, 2021). The interpretability and explainability of THOR comes from a combination of the equation transform at its core (Sonnewald et al, 2019), the engineering of its input features, and the LRP explanation of its predictive skill.…”

“…manuscript submitted to Journal of Advances in Modeling Earth Systems (JAMES) combination of domain knowledge and emerging ML techniques such as AFA could be of pivotal importance for applications within the physical sciences (Sonnewald et al, 2021b;Balaji, 2020;Irrgang et al, 2021;McGovern et al, 2019;Toms et al, 2020). When used as a 'black box', a NN will be trained to make desired prediction, and while it can be skillful in making these predictions, it could have skill rooted in chance more than physics.…”

“…Second, adoption of ML within the physical sciences suffers from a lack of trust that stems from a lack of a transparent understanding of the source of predictive skill (Irrgang et al, 2021;Rudin, 2019;Sonnewald et al, 2021). Ensuring that what is learned by the machine is physically meaningful, and not due to trivial coincidences, is important for example for reliability and generalization (Balaji, 2020), and to avoid underspecification (D'Amour et al, 2020). Trustworthy ML has also been called for in government guidelines from the European Union (Assessment List for Trustworthy Artificial Intelligence) and in a mandate in the United States of America (E.O.…”

Stochastic‐Deep Learning Parameterization of Ocean Momentum Forcing

“…Ideally, increasing the spatio‐temporal resolution could improve these truncated simulations. However, even with the increasing available computational power, running high‐resolution climate models over decades or centuries is not a viable approach within the near future (Balaji, 2021). Typically, the impact of unresolved small‐scale processes on coarse quantities is accounted for via parameterizations.…”