Data Assimilation in the Latent Space of a Convolutional Autoencoder

Amendola, Maddalena; Arcucci, Rossella; Mottet, Laetitia; Casas, César Quilodrán; Fan, Shiwei; Pain, Christopher C.; Linden, P. F.; Guo, Yike

doi:10.1007/978-3-030-77977-1_30

Cited by 23 publications

(48 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…the assumption that the errors follow a Gaussian distribution), or to isolate relevant scales in observational and model states: a ML process can learn to compute the DA correction in optimal space. Some examples of this approach have been developed [7,152,105,87], but so far none of them have been applied to realistic ocean DA setups.…”

Section: Model Errors and ML Within Data Assimilationmentioning

confidence: 99%

Bridging observation, theory and numerical simulation of the ocean using Machine Learning

Sonnewald,

Lguensat,

Jones

et al. 2021

Preprint

View full text Add to dashboard Cite

Section: Model Errors and ML Within Data Assimilationmentioning

confidence: 99%

Bridging observation, theory and numerical simulation of the ocean using Machine Learning

Sonnewald,

Lguensat,

Jones

et al. 2021

Preprint

View full text Add to dashboard Cite

Section: Introductionmentioning

confidence: 99%

“…To incorporate real-time observations for correcting the prediction of the surrogate model, the idea of Latent Assimilation (LA) was introduced [7,8,9] where DA is performed directly in the reduced-order latent space. It has been shown in [7] that LA has a significant advantage in terms of computational efficiency compared to classical fullspace DA methods. However, current approaches of LA require the compression of the observation data into the same latent space of the state variables, which is cumbersome for some applications where the states and the observations are either compressed using different AEs or different physical quantities.…”

Section: Introductionmentioning

confidence: 99%

“…In order to incorporate unseen real-time observation data efficiently, the recent works of [7,8,25] introduce the concept of LA where an AE network is used to compress the state variables and pre-processed observation data. The DA updating is performed in the reduced-order latent space subsequently.…”

Section: Introductionmentioning

confidence: 99%

“…In this study, we develop a novel LA algorithm scheme which generalises the current LA framework [7] to heterogeneous latent spaces and non-linear transformation operators while keeping the important advantage of LA in terms of low computational cost.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Generalised Latent Assimilation in Heterogeneous Reduced Spaces with Machine Learning Surrogate Models

Cheng¹,

Chen²,

Anastasiou³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Reduced-order modelling and low-dimensional surrogate models generated using machine learning algorithms have been widely applied in high-dimensional dynamical systems to improve the algorithmic efficiency. In this paper, we develop a system which combines reduced-order surrogate models with a novel data assimilation (DA) technique used to incorporate real-time observations from different physical spaces. We make use of local smooth surrogate functions which link the space of encoded system variables and the one of current observations to perform variational DA with a low computational cost. The new system, named Generalised Latent Assimilation can benefit both the efficiency provided by the reduced-order modelling and the accuracy of data assimilation. A theoretical analysis of the difference between surrogate and original assimilation cost function is also provided in this paper where an upper bound, depending on the size of the local training set, is given. The new approach is tested on a high-dimensional computational fluid dynamics (CFD) application of a two-phase liquid flow with non-linear observation operators that current Latent Assimilation methods can not handle. Numerical results demonstrate that the proposed assimilation approach can significantly improve the reconstruction and prediction accuracy of the deep learning surrogate model which is nearly 1000 times faster than the CFD simulation.

show abstract

An Automatic Differentiation Method for Surface Carbon Flux Inversion

Yao

Chen

Wang

et al. 2022

Preprint

View full text Add to dashboard Cite

We attempt, for the first time, to estimate the surface carbon flux by the method of automatic differentiation. The atmospheric transport model is developed using a deep learning framework and is validated against standard approaches. Depends on the built-in automatic differentiation feature of the deep learning framework, the system derivatives/gradients are readily available without any extra effort. We then formulate the surface carbon flux estimation as an inverse problem using the variational approach, driven by back-propagated objective gradients. The feasibility of the automatic differentiation method is demonstrated in identical-twin observing system simulation experiments (OSSEs). The proposed framework shows favorable accuracy and great efficiency in both fully and partially observable scenarios. The present study establishes a link between machine learning frameworks and general data assimilation or inverse modeling problems, and the promising results encourage more investigations in incorporating machine learning techniques in inverse carbon modeling.

show abstract

Data Assimilation in the Latent Space of a Convolutional Autoencoder

Cited by 23 publications

References 20 publications

Bridging observation, theory and numerical simulation of the ocean using Machine Learning

Bridging observation, theory and numerical simulation of the ocean using Machine Learning

Generalised Latent Assimilation in Heterogeneous Reduced Spaces with Machine Learning Surrogate Models

An Automatic Differentiation Method for Surface Carbon Flux Inversion

Contact Info

Product

Resources

About