Analog ensemble data assimilation and a method for constructing analogs with variational autoencoders

Abstract

It is proposed to use analogs of the forecast mean to generate an ensemble of perturbations for use in ensemble optimal interpolation (EnOI) or ensemble variational (EnVar) methods. A new method of constructing analogs using variational autoencoders (VAEs; a machine learning method) is proposed. The resulting analog methods using analogs from a catalog (AnEnOI), and using constructed analogs (cAnEnOI), are tested in the context of a multiscale Lorenz‐‘96 model, with standard EnOI and an ensemble square root filter for comparison. The use of analogs from a modestly‐sized catalog is shown to improve the performance of EnOI, with limited marginal improvements resulting from increases in the catalog size. The method using constructed analogs (cAnEnOI) is found to perform as well as a full ensemble square root filter, and to be robust over a wide range of tuning parameters.

A hybrid particle-ensemble Kalman filter for problems with medium nonlinearity

“…There are fewer X k variables, and they evolve more slowly than the Y j , k variables, so the X k variables are typically viewed as ‘large-scale’ while the Y j , k variables are viewed as ‘small-scale.’ The difficulty with this model is that it lacks a clear connection to a spatial field of a physical quantity like temperature or velocity, observations of which contain both large and small scales. A model inspired by the Lorenz-’96 models that possesses a single set of variables x i with distinct large-scale and small-scale dynamics was developed in [ 45 ] and has been used recently as a test model for data assimilation in [ 61 ]. The model is governed by a system of ordinary differential equations of the form where , , 1 is a vector of ones, and The number of state variables in x is 41 J ; here J = 128 for a total system dimension of 5248.…”

Bridging observation, theory and numerical simulation of the ocean using Machine Learning

“…the assumption that the errors follow a Gaussian distribution), or to isolate relevant scales in observational and model states: a ML process can learn to compute the DA correction in optimal space. Some examples of this approach have been developed [7,152,105,87], but so far none of them have been applied to realistic ocean DA setups.…”

An Analog Offline EnKF for Paleoclimate Data Assimilation

“…An optimal selection of ensembles has been widely recognized effective in numerical weather prediction (NWP) for ensemble forecasts (e.g., van den Dool, 1989; Delle Monache et al., 2013; Ren et al., 2020) and also in weather and climate reconstructions (e.g., Devers et al., 2021; Li et al., 2012; Pfister et al., 2020; Samakinwa et al., 2021; Schenk & Zorita, 2012). Analog ensemble data assimilation that uses analogs of forecasts to generate ensembles has also been proposed (e.g., Grooms, 2021; Lguensat et al., 2017; Tandeo et al., 2015). The analog‐based ensemble has the advantage of being able to capture flow‐dependent error statistics that otherwise are missed in the standard offline ensemble‐based PDA methods.…”