Gyrokinetic simulations on many- and multi-core architectures with the global electromagnetic Particle-In-Cell Code ORB5

Abstract

Gyrokinetic codes in plasma physics need outstanding computational resources to solve increasingly complex problems, requiring the effective exploitation of cutting-edge HPC architectures. This paper focuses on the enabling of ORB5, a state-of-the-art, first-principles-based gyrokinetic code, on modern parallel hybrid multi-core, multi-GPU systems. ORB5 is a Lagrangian, Particle-In-Cell (PIC), finite element, global, electromagnetic code, originally implementing distributed parallelism through MPI-based on domain decomposition and domain cloning.In order to support multi/many cores devices, the code has been completely refactored. Data structures have been re-designed to ensure efficient memory access, enhancing data locality. Multi-threading has been introduced through OpenMP on the CPU and adopting OpenACC to support GPU acceleration. MPI can still be used in combination with the two approaches. The performance results obtained using the full production ORB5 code on the Summit system at ORNL, on Piz Daint at CSCS and on the Marconi system at CINECA are presented, showing the effectiveness and performance portability of the adopted solutions: the same source code version was used to produce all results on all architectures.

Orb5: A global electromagnetic gyrokinetic code using the PIC approach in toroidal geometry

“…To take advantage of the new HPC platforms equipped with accelerators, the Orb5 code has been recently ported to GPU using OpenACC. These developments will be detailed in a separate paper [78]. The choice of using OpenMP and OpenACC was motivated because they allow us to keep all options in a single source code version.…”

Numerics and computation in gyrokinetic simulations of electromagnetic turbulence with global particle-in-cell codes

“…A very good scaling of the code has been reported in Ref. [11] on various GPU systems and including electromagnetic perturbations. Our paper extends these results to massive production simulations of electromagnetic turbulence deploying GPUs on the Marconi100 computing system (CINECA).…”

Moment approach of the multi-species non-linear Coulomb collision operator adapted to particle-in-cell codes

“…An alternative to the use of OpenMP has been recently introduced in ORB5 [29] in order to effectively exploit modern GPU accelerated systems. The GPU implementation adopts the OpenACC 2 programming model to enable to the GPU the most time consuming parts of the code.…”