Data assimilation as a deep learning tool to infer ODE representations of dynamical models

Abstract

Recent progress in machine learning has shown how to forecast and, to some extent, learn the dynamics of a model from its output, resorting in particular to neural networks and deep learning techniques. We will show how the same goal can be directly achieved using data assimilation techniques without leveraging on machine learning software libraries, with a view to high-dimensional models. The dynamics of a model are learned from its observation and an ordinary differential equation (ODE) representation of this model is inferred using a recursive nonlinear regression. Because the method is embedded in a 5 Bayesian data assimilation framework, it can learn from partial and noisy observations of a state trajectory of the physical model. Moreover, a space-wise local representation of the ODE system is introduced and is key to cope with high-dimensional models.It has recently been suggested that neural network architectures could be interpreted as dynamical systems. Reciprocally, we show that our ODE representations are reminiscent of deep learning architectures. Furthermore, numerical analysis considera-10 tions on stability shed light on the assets and limitations of the method.The method is illustrated on several chaotic discrete and continuous models of various dimensions, with or without noisy observations, with the goal to identify or improve the model dynamics, build a surrogate or reduced model, or produce forecasts from mere observations of the physical model.

An algorithm for non-parametric estimation in state–space models

“…Some results are reported in a Supplementary Material document for the sake of brevity. This section focuses on the more challenging 3-dimensional Lorenz model (see in Lorenz, 1963) which is one of the favorite toy models in data assimilation since it is a sophisticated (nonlinear, non-periodic, chaotic) but low-dimensional dynamical system (Lguensat et al, 2017;Bocquet et al, 2019). The considered Lorenz-63 (L63) SSM on R 3 is defined as…”

“…Beskos et al, 2017) with advanced machine learning approaches for estimating m (see e.g. Bocquet et al, 2019;Fablet et al, 2017a) may allow to tackle higher dimensional problems with the additional advantage of leading to a reduction of computational costs if the machine learning tool is efficiently implemented.…”

“…In Lguensat et al (2017), better results were obtained with a slightly more sophisticated estimator known as local linear regression (LLR) in statistics (Cleveland and Devlin, 1988) and constructed analogs in meteorology (Tippett and DelSole, 2013). More recently, it has been proposed to use other machine learning (ML) tools such as deep learning (see Bocquet et al, 2019) or sparse regression (see Brunton et al, 2017) to better handle high dimensional data and mimic the behavior of numerical methods used to approximate the solutions of physical models.…”

Bridging observation, theory and numerical simulation of the ocean using Machine Learning

“…5.2.3. Data assimilation and ML Several studies have highlighted the connection between DA and ML [1,30,36,95]. The connection is more direct with 4DVar, in which a function that quantifies model-data disagreement (i.e.…”

Statistical Post-Processing for Gridded Temperature Prediction Using Encoder–Decoder-Based Deep Convolutional Neural Networks

“…Recent advancements in machine learning technologies, particularly deep learning (DL) or deep neural networks, have largely impacted various industries due to precise image and speech recognition capability (Alam et al 2020;Yu et al 2020). Many meteorological studies based on DL have been conducted in recent years, including the discrimination of typhoon intensity (Pradhan et al 2018;Wimmers 2019;Daikoji et al 2020), precipitation nowcasting (Shi et al 2015;Samsi et al 2019;Agrawal et al 2019), quality check algorisms for observed values (Dai et al 2018), data assimilation (Bocquet et al 2019;Arcucci et al 2021), and subgrid parameterizations (Rasp et al 2018;Han et al 2020). However, only a few studies based on DL have been conducted on guidance for short-range weather forecasts (Scheuerer et al 2020;Veldkamp et al 2021), despite it has used machine learning techniques for approximately 50 years (Glahn and Lowry 1972).…”