Towards physically consistent data-driven weather forecasting: Integrating data assimilation with equivariance-preserving spatial transformers in a case study with ERA5

Abstract

Abstract. There is growing interest in data-driven weather prediction (DDWP), for example using convolutional neural networks such as U-NETs that are trained on data from models or reanalysis. Here, we propose 3 components to integrate with commonly used DDWP models in order to improve their physical consistency and forecast accuracy. These components are 1) a deep spatial transformer added to the latent space of the U-NETs to preserve a property called equivariance, which is related to correctly capturing rotations and scalings of features in spatio-temporal data, 2) a data-assimilation (DA) algorithm to ingest noisy observations and improve the initial conditions for next forecasts, and 3) a multi-time-step algorithm, which combines forecasts from DDWP models with different time steps through DA, improving the accuracy of forecasts at short intervals. To show the benefit/feasibility of each component, we use geopotential height at 500 hPa (Z500) from ERA5 reanalysis and examine the short-term forecast accuracy of specific setups of the DDWP framework. Results show that the equivariance-preserving networks (U-STNs) clearly outperform the U-NETs, for example improving the forecast skill by 45 %. Using a sigma-point ensemble Kalman (SPEnKF) algorithm for DA and U-STN as the forward model, we show that stable, accurate DA cycles are achieved even with high observation noise. The DDWP+DA framework substantially benefits from large (O(1000)) ensembles that are inexpensively generated with the data-driven forward model in each DA cycle. The multi-time-step DDWP+DA framework also shows promises, e.g., it reduces the average error by factors of 2–3. These results show the benefits/feasibilities of these 3 components, which are flexible and can be used in a variety of DDWP setups. Furthermore, while here we focus on weather forecasting, the 3 components can be readily adopted for other parts of the Earth system, such as ocean and land, for which there is a rapid growth of data and need for forecast/assimilation.

Building a Better Forecast: Reformulating the Ensemble Kalman Filter for Improved Applications to Volcano Deformation

“…This approach, however, comes at the cost of lower precision and may not be compatible with all implementations of the update algorithm. Looking forward, researchers in other fields have begun experimenting with machine learning, outsourcing at least some computations to neural networks trained on a large set of model realizations (e.g., Chattopadhyay et al., 2021 ). However, this technique is relatively new and has not yet been adapted for this application.…”

Updates on Model Hierarchies for Understanding and Simulating the Climate System: A Focus on Data‐Informed Methods and Climate Change Impacts

“…Physics‐informed ML (or knowledge/theory‐guided ML) couples physical knowledge to the ML architecture and offers one approach to enhance ML generalizability and trust (e.g., Gentine et al., 2021; Irrgang et al., 2021; Karpatne et al., 2017; Kashinath et al., 2021; Raissi et al., 2019a). Here, physical conservation laws are incorporated into ML algorithms, either by constraining the loss function (soft constraints, also called regularization; e.g., Brenowitz et al., 2020; Harder et al., 2022) or by more strictly enforcing conserved properties (hard constraints; e.g., Beucler et al., 2021a; Chattopadhyay et al., 2021). Variations of physics‐informed ML include designing “climate‐invariant” algorithms by rescaling inputs and outputs to avoid extrapolation (Beucler et al., 2021b) or incorporating equations governing the dynamics to build hybrid ML algorithms (e.g., Pathak et al., 2018b; Raissi et al., 2019a).…”

“…These methods both involve careful design based on expert knowledge of the possible regimes within the climate system, much like a traditional physics‐based model, and once built, can also harness the computational speed‐ups offered by ML. Similarly, physics‐informed or hybrid approaches as discussed above offer a way to incorporate interpretable steps into a model (e.g., by enforcing conservation laws, Beucler et al., 2021a; Chattopadhyay et al., 2021). Alternatively, techniques such as data‐driven equation discovery can learn closed‐form equations for SGS parameterizations, which are more easily interpreted (e.g., Mojgani et al., 2022; Raissi et al., 2019b; Zanna & Bolton, 2020).…”

A Four‐Dimensional Variational Constrained Neural Network‐Based Data Assimilation Method

“…Furthermore, a sigma‐point ensemble Kalman algorithm and the U‐STN model were also integrated in Chattopadhyay et al. (2021) to provide stable, accurate DA cycles for geopotential height prediction. It showed that the gain from applying DA to an ML‐based surrogate model would be most significant when the observations are noisy and sparse.…”