The FastEddy® Resident‐GPU Accelerated Large‐Eddy Simulation Framework: Model Formulation, Dynamical‐Core Validation and Performance Benchmarks

Abstract

This paper introduces a new large-eddy simulation model, FastEddy ® , purpose built for leveraging the accelerated and more power-efficient computing capacity of graphics processing units (GPUs) toward adopting microscale turbulence-resolving atmospheric boundary layer simulations into future numerical weather prediction activities. Here a basis for future endeavors with the FastEddy ® model is provided by describing the model dry dynamics formulation and investigating several validation scenarios that establish a baseline of model predictive skill for canonical neutral, convective, and stable boundary layer regimes, along with boundary layer flow over heterogeneous terrain. The current FastEddy ® GPU performance and efficiency gains versus similarly formulated, state-of-the-art CPU-based models is determined through scaling tests as 1 GPU to 256 CPU cores. At this ratio of GPUs to CPU cores, FastEddy ® achieves 6 times faster prediction rate than commensurate CPU models under equivalent power consumption. Alternatively, FastEddy ® uses 8 times less power at this ratio under equivalent CPU/GPU prediction rate. The accelerated performance and efficiency gains of the FastEddy ® model permit more broad application of large-eddy simulation to emerging atmospheric boundary layer research topics through substantial reduction of computational resource requirements and increase in model prediction rate. Plain Language Summary This paper introduces a new model for atmospheric flows, FastEddy ® , engineered to permit faster, more power-efficient, and more broad engagement in simulation of atmospheric flows at high levels of spatial and temporal detail by using graphics cards for accelerating computations. A model description and set of pertinent validation efforts are provided along with performance intercomparison versus two state-of-the-art and widely used models of a similar vein. The documentation of formulation and validity along with the accelerated performance and more power-efficient capability of FastEddy ® provides a comprehensive and robust basis for future adoption and extension as an enabling technology for high-impact atmospheric boundary layer research and applications.

Large-Eddy Simulations of Stability-Varying Atmospheric Boundary Layer Flow over Isolated Buildings

“…The NCAR GPU-accelerated FastEddy LES model (Sauer and Muñoz-Esparza 2020) used in the present study solves the fully compressible Navier-Stokes equations discretized using finite difference approximations. A third-order in time Runge-Kutta scheme and an upwind-biased fifth-order in space advection scheme on an unstaggered A grid were used unless otherwise stated.…”

“…The surface momentum and heat fluxes at ground surfaces were parameterized based on the Monin-Obukhov similarity theory (Monin and Obukhov 1954), with stability correction terms (Dyer and Hicks 1970;Dyer 1974). The reader is referred to Sauer and Muñoz-Esparza (2020) for a comprehensive description of the FastEddy model formulation, dynamical core validation, and performance benchmarks.…”

“…A number of LES studies have contributed to advancing the knowledge on this subject mainly for the flow over horizontally homogeneous and structure-free surfaces (e.g., Nieuwstadt et al 1993;Andren et al 1994;Moeng and Sullivan 1994;Kosović and Curry 2000;Kumar et al 2010;Sullivan et al 2016;Salesky et al 2017). Recently, the LES scope has been expanded to include the flow over urban landscapes and/or complex terrains with improvements to the explicit representation of effects of these heterogeneous surfaces, in response to an increasing demand for microscale information of the urban flows (e.g., Bou-Zeid et al 2009;Lundquist et al 2012;Giometto et al 2016Giometto et al , 2017Gronemeier et al 2017;García-Sánchez et al 2018;Auguste et al 2019;Auvinen et al 2020;Baumman-Stanzer et al 2020;Sauer and Muñoz-Esparza 2020;Muñoz-Esparza et al 2020). However, the majority of these urban-resolving LES studies have focused on neutrally stratified ABL (NBL) conditions.…”

“…As a first step toward bridging this gap in the fundamental understanding of building-induced local flow structures in realistic ABL stability regimes, we explore the response of flow around isolated cuboid buildings under various atmospheric stability conditions, modulated by changing the surface heating and vertical structure of the incoming flow, which is at a quasi-equilibrium state for the surface heating. The NCAR GPU-accelerated LES model, FastEddy Ò (hereafter FastEddy) (Sauer and Muñoz-Esparza 2020), is used with explicit representation of building effects using an immersed body force method (IBFM; Muñoz-Esparza et al 2020) and a Smagorinsky-Lilly subgrid-scale (SGS) model (Lilly 1966;Deardorff 1980) based on a diagnostic SGS turbulence kinetic energy (TKE) equation. The computational efficiency gain of the GPU-based FastEddy LES model compared to CPU-based models, 1 GPU matching 256 CPUs (Sauer and Muñoz-Esparza 2020), enables a comprehensive assessment based on a large number of buildingresolving simulations: a total of 33 simulations with 21 of them at 1-or 2-m grid spacing.…”

Inclusion of Building‐Resolving Capabilities Into the FastEddy® GPU‐LES Model Using an Immersed Body Force Method

“…In that regard, a novel resident-GPU large-eddy simulation (LES) model for the study of atmospheric boundary layer (ABL) phenomena: FastEddy ® (hereafter referred to as FastEddy), was recently introduced. Model formulation and implementation details of FastEddy are described in Sauer and Muñoz-Esparza (2020), together with a suite of validation cases including canonical ABL flows for different atmospheric stabilities and flow over complex terrain. FastEddy has proven to be highly efficient, demonstrating performance speed up of 1 GPU matching 256 CPU cores (6 times faster prediction rate under equivalent power or 8 times lower power consumption at an equivalent CPU/GPU prediction rate).…”