A Physics-Informed Data-Driven Algorithm for Ensemble Forecast of Complex Turbulent Systems

Abstract

A new ensemble forecast algorithm, named as the physics-informed data-driven algorithm with conditional Gaussian statistics (PIDD-CG), is developed to predict the time evolution of the probability density functions (PDFs) of complex turbulent systems with partial observations. The PIDD-CG algorithm integrates a unique multiscale statistical closure model with an extremely efficient nonlinear data assimilation scheme to represent the PDF as a mixture of conditional statistics, which overcomes the curse of dimensionality for high-dimensional systems. The multiscale features in the time evolution of each conditional statistics ensemble member effectively captured by an appropriate combination of physics-informed analytic formulae and recurrent neural networks. An information metric is adopted as the loss function for the latter to more accurately calibrate the key turbulent signals with strong fluctuations. The proposed algorithm succeeds in forecasting both the transient and statistical equilibrium non-Gaussian PDFs of strongly turbulent systems with intermittency, regime switching and extreme events.

Reduced-Order Autodifferentiable Ensemble Kalman Filters

“…In contrast, our paper introduces a framework that uses data assimilation as a tool to build surrogate latent dynamics from data. Another approach to reduce the dimension of data assimilation problems exploits the conditionally Gaussian distribution of slow variables arising in the stochastic parameterization of a wide range of dynamical systems [17,18,61]. This conditional Gaussian structure can be exploited to obtain adequate uncertainty quantification of forecasts with a moderate sample size.…”

“…A caveat, however, is that identifying the slow variables can be challenging in practice. As in our approach, these techniques often rely on machine learning to learn closure terms for the dynamics [18]. Finally, we refer to [78] for a discussion on how the effective dimension of transport map methods for data assimilation can be reduced by exploiting the conditional independence structure of the reference-target pair.…”

Combining Stochastic Parameterized Reduced‐Order Models With Machine Learning for Data Assimilation and Uncertainty Quantification With Partial Observations

“…Stochastic parameterizations are often incorporated into the resulting reduced-order models to further improve the computational efficiency and accuracy (Berner et al, 2017;Christensen et al, 2017;Duan & Nadiga, 2007;Grooms et al, 2015;Mana & Zanna, 2014;Schneider et al, 2021). On the other hand, as the primary goal for the time integration of the model is to seek the forecast statistics, nonlinear statistical reduced-order models have been developed to predict the leading few moments that help reconstruct the forecast PDF (Chen & Qi, 2022;A. J. Majda et al, 2014;Qi & Harlim, 2022;Sapsis & Majda, 2013).…”