Data Assimilation in the Latent Space of a Neural Network

Abstract

There is an urgent need to build models to tackle Indoor Air Quality issue. Since the model should be accurate and fast, Reduced Order Modelling technique is used to reduce the dimensionality of the problem. The accuracy of the model, that represent a dynamic system, is improved integrating real data coming from sensors using Data Assimilation techniques. In this paper, we formulate a new methodology called Latent Assimilation that combines Data Assimilation and Machine Learning. We use a Convolutional neural network to reduce the dimensionality of the problem, a Long-Short-Term-Memory to build a surrogate model of the dynamic system and an Optimal Interpolated Kalman Filter to incorporate real data. Experimental results are provided for CO 2 concentration within an indoor space. This methodology can be used for example to predict in real-time the load of virus, such as the SARS-COV-2, in the air by linking it to the concentration of CO 2 .

Observation Error Covariance Specification in Dynamical Systems for Data assimilation using Recurrent Neural Networks

“…Future research should also consider applying the new method to a broader range of real-world problems, including NWP, hydrology, and object tracking, where the offline data simulation could be more computationally expensive compared to the two test models presented in this paper. To this end, future studies could also investigate the combination of model reduction methods, such as domain localization [56], proper orthogonal decomposition, information-based data compression [57], auto-encoder neural networks [58], and the current covariance estimation method. More precisely, the data assimilation can be performed in the compressed low dimensional space (e.g., obtained from POD or auto-encoder).…”

Equation‐Free Surrogate Modeling of Geophysical Flows at the Intersection of Machine Learning and Data Assimilation

“…However, in this study, the forward model is replaced with the surrogate model, and therefore, we perform the DA in the latent space. The similar ideas have also been used in other studies that deals with the DA for reduced-order models (Amendola et al, 2020;Maulik et al, 2022;Peyron et al, 2021). Once the LSTM network is trained, it is used to forecast the future state of the POD modal coefficients in an auto-regressive manner .…”

An efficient digital twin based on machine learning SVD autoencoder and generalised latent assimilation for nuclear reactor physics

“…Further more, because of the high-efficiency of the machine learning based ROM forward model, when constructing the inverse model to infer the input parameter with online sensor data, we proposed a relatively naive approach in [36] to approximate the model parameters from an ensemble of samplings generated using LHS around the initial guess. Meanwhile, the latest progress of data assimilation in latent space [61,1,16,27,52] (also called latent assimilation) with machine learning make it possible to further simplify our inverse problem instead of the naive sampling approach. Thus our second contribution in this work it to adopt the generalised latent assimilation to solve the inverse problem, which ensures the high efficiency of the operation digital twin.…”

“…Data-driven models such as data assimilation (DA) approaches can be applied [5,23]. Recently, latent assimilation techniques are introduced in the work of [1,61,27] where the DA is performed after having compressed the state and the observation data into the same latent space. However, as noted in the work [25], it is almost infeasible to compress the full state space and observations into a same latent space in a wide range of DA applications, where only a part of the state are observable.…”