Automatic Code Generation for High-performance Discontinuous Galerkin Methods on Modern Architectures

Abstract

SIMD vectorization has lately become a key challenge in high-performance computing. However, hand-written explicitly vectorized code often poses a threat to the software’s sustainability. In this publication, we solve this sustainability and performance portability issue by enriching the simulation framework dune-pdelab with a code generation approach. The approach is based on the well-known domain-specific language UFL but combines it with loopy, a more powerful intermediate representation for the computational kernel. Given this flexible tool, we present and implement a new class of vectorization strategies for the assembly of Discontinuous Galerkin methods on hexahedral meshes exploiting the finite element’s tensor product structure. The performance-optimal variant from this class is chosen by the code generator through an auto-tuning approach. The implementation is done within the open source PDE software framework Dune and the discretization module dune-pdelab. The strength of the proposed approach is illustrated with performance measurements for DG schemes for a scalar diffusion reaction equation and the Stokes equation. In our measurements, we utilize both the AVX2 and the AVX512 instruction set, achieving 30% to 40% of the machine’s theoretical peak performance for one matrix-free application of the operator.

Enhancing data locality of the conjugate gradient method for high-order matrix-free finite-element implementations

“…For tensor-product shape functions that are integrated on a tensor-product quadrature formula, sum factorization allows to decompose these two steps into a series of one-dimensional interpolations of total cost O(pd+1) per element in d dimensions (or O(p) per unknown), compared to the naive evaluation cost of O(p2d). The sum-factorization approach has been developed in the context of the spectral element method by Orszag (1980), Patera (1984), and Tufo and Fischer (1999), see also the book by Deville et al (2002) as well as recent implementation and vectorization studies by Kronbichler and Kormann (2012, 2019), Świrydowicz et al (2019), Fischer et al (2020), Sun et al (2020), Moxey et al (2020), and Kempf et al (2021).…”

Enhancing data locality of the conjugate gradient method for high-order matrix-free finite-element implementations

“…The sum-factorization approach has been developed in the context of the spectral element method by Orszag (1980), Patera (1984), and Tufo and Fischer (1999), see also the book by Deville et al (2002) as well as recent implementation and vectorization studies by Kormann (2012, 2019), Świrydowicz et al (2019Świrydowicz et al ( ), Fischer et al (2020, Sun et al (2020), Moxey et al (2020), and Kempf et al (2021).…”

A next-generation discontinuous galerkin fluid dynamics solver with application to high-resolution lung airflow simulations

“…Our highly-tuned kernels use Flop-minimizing optimizations in the sum factorization algorithms, such as change of basis and even-odd decomposition [43]. These optimizations yield a speedup of 1.5×-2× compared to previous results such as [38]. Cell and face integrals are computed alternately, in order to limit the access in the source vector P to a single access from main memory and serve the other accesses primarily from caches.…”