Learning physics-constrained subgrid-scale closures in the small-data regime for stable and accurate LES

Abstract

We demonstrate how incorporating physics constraints into convolutional neural networks (CNNs) enables learning subgrid-scale (SGS) closures for stable and accurate large-eddy simulations (LES) in the small-data regime (i.e., when the availability of high-quality training data is limited). Using several setups of forced 2D turbulence as the testbeds, we examine the a priori and a posteriori performance of three methods for incorporating physics: 1) data augmentation (DA), 2) CNN with group convolutions (GCNN), and 3) loss functions that enforce a global enstrophy-transfer conservation (EnsCon). While the data-driven closures from physicsagnostic CNNs trained in the big-data regime are accurate and stable, and outperform dynamic Smagorinsky (DSMAG) closures, their performance substantially deteriorate when these CNNs are trained with 40x fewer samples (the small-data regime). An example based on a vortex dipole demonstrates that the physics-agnostic CNN cannot account for never-seen-before samples ′ rotational equivariance (symmetry), an important property of the SGS term. This shows a major shortcoming of the physics-agnostic CNN in the small-data regime. We show that CNN with DA and GCNN address this issue and each produce accurate and stable data-driven closures in the small-data regime. Despite its simplicity, DA, which adds appropriately rotated samples to the training set, performs as well or in some cases even better than GCNN, which uses a sophisticated equivariance-preserving architecture. EnsCon, which combines structural modeling with aspect of functional modeling, also produces accurate and stable closures in the small-data regime. Overall, GCNN+EnCon, which combines these two physics constraints, shows the best a posteriori performance in this regime. These results illustrate the power of physics-constrained learning in the small-data regime for accurate and stable LES.

Explaining the physics of transfer learning a data-driven subgrid-scale closure to a different turbulent flow

“…(1a)-(1b) are solved using a Fourier-Fourier pseudo-spectral solver with N DNS collocation grid points and second-order Adams-Bashforth and Crank-Nicolson time-integration schemes with time step ∆t DNS for the advection and viscous terms, respectively. See Guan et al [22,40] for more details on the solvers and these simulations. For the base system in Case 1 (decaying 2D turbulence), following earlier studies [22,37], the flow is initialized randomly using a vorticity field (ω ic ) with a prescribed power spectrum.…”

“…A.2 Filtering and coarse-graining: LES equations and FDNS data Filtering Eqs. (1a)-(1b) yields the governing equations for LES [40,48,53]:…”

“…r is the linear drag coefficient and f (x, y) is a time-independent external forcing at wavenumbers m f and n f . This system, with different combinations of f and r, is a fitting prototype for a variety of large-scale geophysical and environmental flows and has been widely used to test novel techniques including data-driven SGS closures [11,22,[37][38][39][40].…”

“…By changing Re, r, m f , and n f , we have created 6 distinctly different flows, divided into 3 cases, each with a base and a target system (Table 1 and Section A.3). We have shown in previous studies that for various setups of 2D turbulence, CNNs trained on large training sets, or on small training sets with physics-constraints incorporated, produce accurate and stable data-driven closures in a priori (offline) and a posteriori (online) tests [22,40]. These CNN-based closures were found to accurately capture both diffusion and backscattering, and to outperform widely used physics-based SGS closures such as the Smagorinsky, dynamic Smagorinsky, and mixed models in both a priori and a posteriori tests.…”

Deep Learning-Enhanced Ensemble-Based Data Assimilation for High-Dimensional Nonlinear Dynamical Systems

“…Recent successes in data-driven short-term weather modeling [53,29,44] show that short-term accurate surrogates can be as accurate as operational numerical models if trained on observations [44]. Further improvement in data-driven surrogates can be achieved by conserving key physics as can be seen in many different applications in both fluids [54] and weather and climate community [43].…”