A mass- and energy-conserving framework for using machine learning to speed computations: a photochemistry example

Sturm, Patrick Obin; Wexler, Anthony S.

doi:10.5194/gmd-13-4435-2020

Cited by 17 publications

(32 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The output variables are the concentrations of the 12 species at the end of the time step. Photolysis frequencies can themselves be emulated using neural networks (Krasnopolsky et al, 2005;Lagerquist et al, 2021;Sturm and Wexler, 2020) but their calculation is cheap compared to the chemical calculation.…”

Section: Machine Learning (Ml) Neural Network Chemical Solvermentioning

confidence: 99%

An Online-Learned Neural Network Chemical Solver for Stable Long-Term Global Simulations of Atmospheric Chemistry

Kelp¹,

Jacob²,

Lin³

et al. 2022

Preprint

View full text Add to dashboard Cite

A major computational barrier in global modeling of atmospheric chemistry is the numerical integration of the coupled kinetic equations describing the chemical mechanism. Machine-learned (ML) solvers can offer order-of-magnitude speedup relative to conventional implicit solvers, but past implementations have suffered from fast error growth and only run for short simulation times (<1 month). A successful ML solver for global models must avoid error growth over year-long simulations and allow for re-initialization of the chemical trajectory by transport at every time step. Here we explore the capability of a neural network solver equipped with an autoencoder to achieve stable full-year simulations of tropospheric oxidant chemistry in the global 3-D GEOS-Chem model, replacing its standard mechanism (228 species) by the Super-Fast mechanism (12 species) to avoid the curse of dimensionality. We find that online training of the ML solver within GEOS-Chem is essential for accuracy, whereas offline training from archived GEOS-Chem inputs/outputs produces large errors. After online training we achieve stable 1-year simulations with five-fold speedup compared to the standard implicit Rosenbrock solver with global tropospheric normalized mean biases of -0.3% for ozone, 1% for hydrogen oxide radicals, and -5% for nitrogen oxides. The ML solver captures the diurnal and synoptic variability of surface ozone at polluted and clean sites. There are however large regional biases for ozone and NOx under remote conditions where chemical aging leads to error accumulation. These regional biases remain a major limitation for practical application, and ML emulation would be more difficult in a more complex mechanism.

show abstract

Section: Machine Learning (Ml) Neural Network Chemical Solvermentioning

confidence: 99%

An Online-Learned Neural Network Chemical Solver for Stable Long-Term Global Simulations of Atmospheric Chemistry

Kelp¹,

Jacob²,

Lin³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Our prior work (Sturm and Wexler, 2020) introduced a framework that could be used with any machine learning algorithm to introduce conservation laws. In the case of atmospheric chemistry, most ML surrogate model approaches have estimated future concentrations 𝑪(𝑡 + ∆𝑡) from current concentrations 𝑪(𝑡) and other parameters 𝑴(𝑡) , which can include meteorological conditions such as zenith angle, temperature, and humidity.…”

Section: Physical Constraints In the Neural Network Architecturementioning

confidence: 99%

“…Unfortunately 𝑺 values cannot be readily gleaned from the reference model for training a machine learning tool, especially when more sophisticated integrators are used. Our prior work focused on a way to invert 𝐀 in order to calculate the target values 𝑺 (Sturm and Wexler, 2020). For the example of a surrogate model of condensation/evaporation in a sectional aerosol model, 𝐀 is overdetermined and a left pseudoinverse exists (see Appendix A1).…”

Section: 𝑪(𝑡mentioning

confidence: 99%

“…Keller and Evans (2019) also provide the example of atom conservation and propose inclusion of stoichiometric information as a possible solution and a future direction to explore. In this work, we focus on this latter goal: conserving atoms, much in line with the suggestions outlined in Keller and Evans (2019), as well as the framework introduced by our prior work (Sturm and Wexler, 2020). More specifically, we utilize the weight matrix multiplication structure of a neural network (NN) to incorporate stoichiometric information in its architecture.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Conservation laws in a neural network architecture: Enforcing the atom balance of a Julia-based photochemical model (v0.2.0)

Sturm

Wexler

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Abstract. Models of atmospheric phenomena provide insight into climate, air quality, and meteorology, and provide a mechanism for understanding the effect of future emissions scenarios. To accurately represent atmospheric phenomena, these models consume vast quantities of computational resources. Machine learning (ML) techniques such as neural networks have the potential to emulate compute-intensive components of these models to reduce their computational burden. However, such ML surrogate models may lead to nonphysical predictions that are difficult to uncover. Here we present a neural network architecture that enforces conservation laws. Instead of simply predicting properties of interest, a physically interpretable hidden layer within the network predicts fluxes between properties which are subsequently related to the properties of interest. As an example, we design a physics-constrained neural network surrogate model of photochemistry using this approach and find that it conserves atoms as they flow between molecules to machine precision, while outperforming a naïve neural network in terms of accuracy and non-negativity of concentrations.

show abstract

“…Recent work took a first step to incorporate fundamental principles and input-output relationships into AI/ML-based emulators 1,[6][7] . In one of these works 1 the AI/ML emulators memorize the fluxes instead of the quantities so that conservation principles are automatically obeyed and applied this to an atmospheric chemistry example.…”

mentioning

confidence: 99%

High-Accuracy Module Emulators from Physically-Constrained AI Algorithms

Wexler

2021

View full text Add to dashboard Cite

How do we use AI tools to integrate observations, simulated data and physical and chemical fundamentals (Focal Area 3) into model components (Focal Area 2) that have high accuracy and stability and low computational burden to improve Earth System Predictability? Science ChallengeEarth system modeling of the hydrological cycle involves compute-intensive modules representing complex chemical and physical process. Recently, AI tools that are far less compute intensive have been developed that emulate these modules, but many of these efforts are not yet sufficiently accurate or even stable. We know a lot about the physics and chemistry of earth system processes. The Science Challenge is developing AI tools that not only incorporate observations and simulated data, but also incorporate the physics and chemistry of the process, while still maintaining the compute efficiency. RationalePredicting extreme events in the hydrological cycle will involve high spatial resolution earth system models. The grand challenge is running global earth system models at a horizontal grid-spacing of a few hundred meters comparable to vertical resolution (rather than several hundred kms, used currently), so that fine-scale processes of aerosol and cloud microphysics, and convection and entrainment can be explicitly resolved. In many earth system models, the compute limitation resides with three modules, namely atmospheric chemistry, aerosol dynamics and radiative transfer, while the wall-clock limitation resides with dynamics and transport. Increasing horizontal spatial resolution to be comparable to the vertical resolution will dramatically increase the compute resources consumed by all of these modules. One path forward is to replace the three modules with machine-learned emulators. Current modules are based on the physics and chemistry of processes, even if they are highly parameterized. Machine-learned module replacements typically memorize data generated by the original module, achieving dramatic computational efficiency improvements, but frequently at the expense of accuracy, stability or both. New machine learning frameworks are needed that integrate all of our knowledge, including fundamental physics and chemistry along with observations or generated data within earth system models. The first such frameworks have recently been developed 1 .

show abstract

A mass- and energy-conserving framework for using machine learning to speed computations: a photochemistry example

Cited by 17 publications

References 17 publications

An Online-Learned Neural Network Chemical Solver for Stable Long-Term Global Simulations of Atmospheric Chemistry

An Online-Learned Neural Network Chemical Solver for Stable Long-Term Global Simulations of Atmospheric Chemistry

Conservation laws in a neural network architecture: Enforcing the atom balance of a Julia-based photochemical model (v0.2.0)

High-Accuracy Module Emulators from Physically-Constrained AI Algorithms

Contact Info

Product

Resources

About