Large-Scale direct numerical simulations of turbulence using GPUs and modern Fortran

Karp, Martin; Massaro, Daniele; Jansson, Niclas; Hart, Alistair; Wahlgren, Jacob; Schlatter, Philipp

doi:10.1177/10943420231158616

Cited by 9 publications

(7 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…32 Among the main conclusions from these works, two deserve special attention. First, papers 5,36,38 demonstrated that the kernels performing well on NVIDIA GPUs do not necessarily map well onto AMD GPUs after porting to HIP, so code optimizations are required to improve the overall performance of HIP codes. The second conclusion refers to the performance constraints of AMD MI200 series GPUs for memory-bound applications like that studied in this paper.…”

Section: Related Workmentioning

confidence: 99%

“…In particular, Kolev et al 38 observed that one logical GPU of the AMD MI250X performs closer to 79% of one NVIDIA A100 in their finite element simulations. According to Karp et al, 5 this performance ratio could be increased in favor of the AMD GPUs-in their turbulence simulations, two logical GPUs in the MI250X correspond to two A100 GPUs with regards to performance. At the same time, it is worth noting that the theoretical peak performance of the MI250X is more than two times higher than two A100 GPUs.…”

Section: Related Workmentioning

confidence: 99%

“…In the past, software efforts primarily concentrated 32 on NVIDIA GPUs due to the mature CUDA development environment and the widespread adoption in the HPC community. However, with the launching of new AMD Instinct series GPUs and especially powerful Instinct MI200 accelerators with the CDNA2 architecture, 10 developers of scientific software have started porting their codes to AMD GPUs 5,33–38 using the HIP programming interface 32 . Among the main conclusions from these works, two deserve special attention.…”

Section: Related Workmentioning

confidence: 99%

“…General purpose GPU computing 1–3 has become a principal trend due to the constantly growing demand for computation capability in important application domains such as large‐scale simulations, deep learning, and big data 4–7 . After a long period of NVIDIA dominance, 8 now real competition among vendors has begun.…”

Section: Introductionmentioning

confidence: 99%

“…Benchmarking the application performance on multi-GPU platforms with up to eight accelerators (NVIDIA V100 GPUs or logical GPUs of MI250) confirms the good scalability of CUDA and HIP codes that allow accelerating the simulation up to about seven times. 5. It is shown that OpenMP can provide performance compared to CUDA and HIP for single GPU with overheads not exceeding 11% and 20%, respectively.…”

mentioning

confidence: 95%

See 4 more Smart Citations

Single‐ and multi‐GPU computing on NVIDIA‐ and AMD‐based server platforms for solidification modeling application

Halbiniak,

Meyer,

Rojek

2023

Concurrency and Computation

View full text Add to dashboard Cite

SummaryThis work explores the performance of single‐ and multi‐GPU computing on state‐of‐the‐art NVIDIA‐ and AMD‐based server‐class hardware using various programming interfaces to accelerate a real‐world scientific application for solidification modeling based on the phase‐field method. The main computations of this memory‐bound application correspond to 20 stencils computed across grid nodes. We investigate the application's scalability for two basic schemes of organizing computation: without and with hiding data transfers behind computation, combined with using either peer‐to‐peer inter‐GPU data transfers through NVIDIA NVLink and AMD Infinity interconnects or communication over the PCIe and main memory. Among the studied programming interfaces is CUDA, HIP, and OpenMP Accelerator Model. While the first two are designed to write the codes for a specific hardware platform, OpenMP enables code portability between NVIDIA and AMD GPUs. The resulting performance is experimentally assessed on computing platforms containing NVIDIA V100 (up to 8 GPUs) and A100 (one GPU), as well as AMD MI210 (one device) and MI250 (up to 8 logical GPUs).

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 95%

See 3 more Smart Citations

Single‐ and multi‐GPU computing on NVIDIA‐ and AMD‐based server platforms for solidification modeling application

Halbiniak,

Meyer,

Rojek

2023

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

Design of Neko—A Scalable High‐Fidelity Simulation Framework With Extensive Accelerator Support

Jansson,

Karp,

Wahlgren

et al. 2024

Concurrency and Computation

View full text Add to dashboard Cite

Recent trends and advancements in including more diverse and heterogeneous hardware in High‐Performance Computing (HPC) are challenging scientific software developers in their pursuit of efficient numerical methods with sustained performance across a diverse set of platforms. As a result, researchers are today forced to re‐factor their codes to leverage these powerful new heterogeneous systems. We present our design considerations of Neko—a portable framework for high‐fidelity spectral element flow simulations. Unlike prior works, Neko adopts a modern object‐oriented Fortran 2008 approach, allowing multi‐tier abstractions of the solver stack and facilitating various hardware backends ranging from general‐purpose processors, accelerators down to exotic vector processors and Field‐Programmable Gate Arrays (FPGAs). Focusing on the performance and portability of Neko, we describe the framework's device abstraction layer managing device memory, data transfer and kernel launches from Fortran, allowing for a solver written in a hardware‐neutral yet performant way. Accelerator‐specific optimizations are also discussed, with auto‐tuning of key kernels and various communication strategies using device‐aware MPI. Finally, we present performance measurements on a wide range of computing platforms, including the EuroHPC pre‐exascale system LUMI, where Neko achieves excellent parallel efficiency for a large direct numerical simulation (DNS) of turbulent fluid flow using up to 80% of the entire LUMI supercomputer.

show abstract

Deep Reinforcement Learning for the Management of the Wall Regeneration Cycle in Wall-Bounded Turbulent Flows

Cavallazzi,

Guastoni,

Vinuesa

et al. 2024

Flow Turbulence Combust

View full text Add to dashboard Cite

The wall cycle in wall-bounded turbulent flows is a complex turbulence regeneration mechanism that remains not fully understood. This study explores the potential of deep reinforcement learning (DRL) for managing the wall regeneration cycle to achieve desired flow dynamics. To create a robust framework for DRL-based flow control, we have integrated the StableBaselines3 DRL libraries with the open-source direct numerical simulation (DNS) solver CaNS. The DRL agent interacts with the DNS environment, learning policies that modify wall boundary conditions to optimise objectives such as the reduction of the skin-friction coefficient or the enhancement of certain coherent structures’ features. The implementation makes use of the message-passing-interface (MPI) wrappers for efficient communication between the Python-based DRL agent and the DNS solver, ensuring scalability on high-performance computing architectures. Initial experiments demonstrate the capability of DRL to achieve drag reduction rates comparable with those achieved via traditional methods, although limited to short time intervals. We also propose a strategy to enhance the coherence of velocity streaks, assuming that maintaining straight streaks can inhibit instability and further reduce skin-friction. Our results highlight the promise of DRL in flow-control applications and underscore the need for more advanced control laws and objective functions. Future work will focus on optimising actuation intervals and exploring new computational architectures to extend the applicability and the efficiency of DRL in turbulent flow management.

show abstract

Large-Scale direct numerical simulations of turbulence using GPUs and modern Fortran

Cited by 9 publications

References 47 publications

Single‐ and multi‐GPU computing on NVIDIA‐ and AMD‐based server platforms for solidification modeling application

Single‐ and multi‐GPU computing on NVIDIA‐ and AMD‐based server platforms for solidification modeling application

Design of Neko—A Scalable High‐Fidelity Simulation Framework With Extensive Accelerator Support

Deep Reinforcement Learning for the Management of the Wall Regeneration Cycle in Wall-Bounded Turbulent Flows

Contact Info

Product

Resources

About