Training physics-based machine-learning parameterizations with gradient-free ensemble Kalman methods

Abstract

Most machine learning applications in Earth system modeling currently rely on gradientbased supervised learning. This imposes stringent constraints on the nature of the data used for training (typically, residual time tendencies are needed), and it complicates learning about the interactions between machine-learned parameterizations and other components of an Earth system model. Approaching learning about process-based parameterizations as an inverse problem resolves many of these issues, since it allows parameterizations to be trained with partial observations or statistics that directly relate to quantities of interest in long-term climate projections. Here we demonstrate the effectiveness of Kalman inversion methods in treating learning about parameterizations as an inverse problem. We consider two different algorithms: unscented and ensemble Kalman inversion. Both methods involve highly parallelizable forward model evaluations, converge exponentially fast, and do not require gradient computations. In addition, unscented Kalman inversion provides a measure of parameter uncertainty. How learning about parameterizations can be posed as an inverse problem and solved by ensemble Kalman methods is illustrated through the calibration of an eddy-diffusivity mass-flux scheme for subgridscale turbulence and convection, using data generated by large-eddy simulations. We find the algorithms amenable to batching strategies, robust to noise and model failures, and efficient in the calibration of hybrid parameterizations that can include empirical closures and neural networks.

Accelerating large-eddy simulations of clouds with Tensor Processing Units

“…Our LES code and the compute resources available on TPUs enable the generation of large libraries of low-cloud simulations (Shen et al, 2022). These can be used both for quantitatively studying mechanisms underlying low-cloud feedbacks to climate change (Bretherton, 2015) and as training data for parameterizations of low clouds for coarseresolution climate models (Couvreux et al, 2021;Hourdin et al, 2021;Lopez-Gomez et al, 2022). The LES code described here is publicly available for this and similar purposes.…”

Ensemble‐Based Experimental Design for Targeting Data Acquisition to Inform Climate Models

“…The sample stage then samples a posterior distribution with MCMC methods, replacing the computationally expensive model with the cheap emulator. This framework can extend to the learning of data‐driven parameterizations or other non‐parametric functions, such as structural model errors (e.g., M. E. Levine & Stuart, 2021; Lopez‐Gomez et al., 2022; Schneider et al., 2022). Our proposed algorithm builds on CES to incorporate Bayesian experimental design at negligible additional computational expense.…”