Model error correction in data assimilation by integrating neural networks

Zhu, Jiangcheng; Hu, Shuang; Arcucci, Rossella; Xu, Chao; Zhu, Jihong; Guo, Yike

doi:10.26599/bdma.2018.9020033

Cited by 26 publications

(17 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The authors account for temporal details in human motion analysis in [30] by incorporating a Kalman filter into a neural network of LSTM. In [31], the authors suggest a methodology that combines NN and DA for model error correction. Instead, in [32], the authors use DA, in particular Kalman tracking, to speed up any learning-based motion tracking method to real-time and to correct some common inconsistencies in motion tracking methods that are based on the camera.…”

Section: Related Work and Contribution Of The Present Workmentioning

confidence: 99%

Deep Data Assimilation: Integrating Deep Learning with Data Assimilation

et al. 2021

Self Cite

View full text Add to dashboard Cite

In this paper, we propose Deep Data Assimilation (DDA), an integration of Data Assimilation (DA) with Machine Learning (ML). DA is the Bayesian approximation of the true state of some physical system at a given time by combining time-distributed observations with a dynamic model in an optimal way. We use a ML model in order to learn the assimilation process. In particular, a recurrent neural network, trained with the state of the dynamical system and the results of the DA process, is applied for this purpose. At each iteration, we learn a function that accumulates the misfit between the results of the forecasting model and the results of the DA. Subsequently, we compose this function with the dynamic model. This resulting composition is a dynamic model that includes the features of the DA process and that can be used for future prediction without the necessity of the DA. In fact, we prove that the DDA approach implies a reduction of the model error, which decreases at each iteration; this is achieved thanks to the use of DA in the training process. DDA is very useful in that cases when observations are not available for some time steps and DA cannot be applied to reduce the model error. The effectiveness of this method is validated by examples and a sensitivity study. In this paper, the DDA technology is applied to two different applications: the Double integral mass dot system and the Lorenz system. However, the algorithm and numerical methods that are proposed in this work can be applied to other physics problems that involve other equations and/or state variables.

show abstract

Section: Related Work and Contribution Of The Present Workmentioning

confidence: 99%

Deep Data Assimilation: Integrating Deep Learning with Data Assimilation

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…A framework for integration of NN with physical models by Data Assimilation (DA) algorithms is described in [16]: the NNs are iteratively trained when observed data are updated. Unfortunately, this approach presents a limit due to the time complexity of the numerical models involved, which limits the use of the forecast model for large data problems.…”

Section: Related Work and Contribution Of The Present Workmentioning

confidence: 99%

Data Assimilation in the Latent Space of a Neural Network

Amendola,

Arcucci,

Mottet

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

There is an urgent need to build models to tackle Indoor Air Quality issue. Since the model should be accurate and fast, Reduced Order Modelling technique is used to reduce the dimensionality of the problem. The accuracy of the model, that represent a dynamic system, is improved integrating real data coming from sensors using Data Assimilation techniques. In this paper, we formulate a new methodology called Latent Assimilation that combines Data Assimilation and Machine Learning. We use a Convolutional neural network to reduce the dimensionality of the problem, a Long-Short-Term-Memory to build a surrogate model of the dynamic system and an Optimal Interpolated Kalman Filter to incorporate real data. Experimental results are provided for CO 2 concentration within an indoor space. This methodology can be used for example to predict in real-time the load of virus, such as the SARS-COV-2, in the air by linking it to the concentration of CO 2 .

show abstract

“…In recent years, there have been several studies on DA where there is partial information on the statespace model. Some studies on state estimations have proposed combining standard DA procedures and neural networks for when f f f t is unknown (see, e.g., [1,2,3,4]). In addition, as another approach that does not employ neural networks, Hamilton et al [5,6,7] proposed a new filter called the Kalman-Takens filter when f f f t or h h h is unknown.…”

Section: Introductionmentioning

confidence: 99%

Composing a surrogate observation operator for sequential data assimilation

Akita¹,

Miyatake²,

Furihata³

2022

Preprint

View full text Add to dashboard Cite

In data assimilation, state estimation is not straightforward when the observation operator is unknown. This study proposes a method for composing a surrogate operator for a true operator. The surrogate model is improved iteratively to decrease the difference between the observations and the results of the surrogate model, and a neural network is adopted in the process. A twin experiment suggests that the proposed method outperforms approaches that use a specific operator that is given tentatively throughout the data assimilation process.

show abstract

Model error correction in data assimilation by integrating neural networks

Cited by 26 publications

References 15 publications

Deep Data Assimilation: Integrating Deep Learning with Data Assimilation

Deep Data Assimilation: Integrating Deep Learning with Data Assimilation

Data Assimilation in the Latent Space of a Neural Network

Composing a surrogate observation operator for sequential data assimilation

Contact Info

Product

Resources

About