β-Variational autoencoders and transformers for reduced-order modelling of fluid flows

Abstract

Variational autoencoder (VAE) architectures have the potential to develop reduced-order models (ROMs) for chaotic fluid flows. We propose a method for learning compact and near-orthogonal ROMs using a combination of a β-VAE and a transformer, tested on numerical data from a two-dimensional viscous flow in both periodic and chaotic regimes. The β-VAE is trained to learn a compact latent representation of the flow velocity, and the transformer is trained to predict the temporal dynamics in latent space. Using the β-VAE to learn disentangled representations in latent space, we obtain a more interpretable flow model with features that resemble those observed in the proper orthogonal decomposition, but with a more efficient representation. Using Poincaré maps, the results show that our method can capture the underlying dynamics of the flow outperforming other prediction models. The proposed method has potential applications in other fields such as weather forecasting, structural dynamics or biomedical engineering.

Stochastic Latent Transformer: Efficient Modeling of Stochastically Forced Zonal Jets

“…In this context, deep learning methods, particularly autoencoders (Kingma & Welling, 2013; LeCun et al., 2015), have surpassed traditional techniques such as proper orthogonal decomposition (Nikolaidis et al., 2016) in constructing spatial ROMs given to their capacity for nonlinear transformations (Floryan & Graham, 2022; Linot & Graham, 2020). Deep learning has also been employed to model the temporal evolution of turbulent fluid flows, significantly reducing computational costs during inference (Karniadakis et al., 2021; Meuris et al., 2023; Novati et al., 2021; Ravuri et al., 2021; Vinuesa & Brunton, 2022; Yousif et al., 2023)—with efforts combining these methods with autoencoders to evolve dynamics in latent space (Hemmasian & Barati Farimani, 2023; Linot & Graham, 2020; Solera‐Rico et al., 2023; Vlachas et al., 2022) outperforming linear methods such as dynamic‐mode decomposition (Schmid & Sesterhenn, 2008). The aforementioned studies have primarily focused on deterministic nonlinear partial differential equations (PDEs), however in many physical applications, including fluid mechanics, state‐dependent stochastic terms are often included to represent unresolved processes such as small‐scale turbulent eddies.…”