End-to-End Kalman Filter in a High Dimensional Linear Embedding of the Observations

Abstract

Data assimilation techniques are the state-of-the-art approaches in the reconstruction of a spatio-temporal geophysical state such as the atmosphere or the ocean. These methods rely on a numerical model that fills the spatial and temporal gaps in the observational network. Unfortunately, limitations regarding the uncertainty of the state estimate may arise when considering the restriction of the data assimilation problems to a small subset of observations, as encountered for instance in ocean surface reconstruction. These limitations motivated the exploration of reconstruction techniques that do not rely on numerical models. In this context, the increasing availability of geophysical observations and model simulations motivates the exploitation of machine learning tools to tackle the reconstruction of ocean surface variables. In this work, we formulate sea surface spatio-temporal reconstruction problems as state space Bayesian smoothing problems with unknown augmented linear dynamics. The solution of the smoothing problem, given by the Kalman smoother, is written in a differentiable framework which allows, given some training data, to optimize the parameters of the state space model.

Machine learning with data assimilation and uncertainty quantification for dynamical systems: a review

“…For instance, when considering noise-free observations, a 4D-var formulation was used in [285], [286] to derive nonlinear dynamical models from partial observations of the state space. In related works, a KFbased identification was proposed for linear dynamical and observation models with Gaussian uncertainties [287], [288].…”