Enhancing data locality of the conjugate gradient method for high-order matrix-free finite-element implementations

Abstract

This work investigates a variant of the conjugate gradient (CG) method and embeds it into the context of high-order finite-element schemes with fast matrix-free operator evaluation and cheap preconditioners like the matrix diagonal. Relying on a data-dependency analysis and appropriate enumeration of degrees of freedom, we interleave the vector updates and inner products in a CG iteration with the matrix-vector product with only minor organizational overhead. As a result, around 90% of the vector entries of the three active vectors of the CG method are transferred from slow RAM memory exactly once per iteration, with all additional access hitting fast cache memory. Node-level performance analyses and scaling studies on up to 147k cores show that the CG method with the proposed performance optimizations is around two times faster than a standard CG solver as well as optimized pipelined CG and s-step CG methods for large sizes that exceed processor caches, and provides similar performance near the strong scaling limit.

A matrix‐free macro‐element variant of the hybridized discontinuous Galerkin method

“…We first assess the efficiency of the global solver, comparing the matrix‐based approach and the matrix‐free approach, see References 49‐52. In the matrix‐based approach, the global system $$$ \left(\mathbf{D}-\mathbf{C}{\mathbf{A}}^{-1}\mathbf{B}\right)\hat{\mathbf{u}}=\mathbf{f} $$$ must be explicitly formed at the macro‐element level, assembled and stored at the global level, and then solved via an iterative solver.…”

Exploiting high‐contrast Stokes preconditioners to efficiently solve incompressible fluid–structure interaction problems

“…We base our implementation on the deal.II library, 65 taking advantage of the specific structure of the linear problem, and leveraging the properties of modern computer hardware, such as the availability of vectorized SIMD instructions and parallelism—by choosing to implement the preconditioner (and the solver) using the matrix‐free approach, 42 which is very well supported in the deal.II library. The CPU cache efficiency is improved because the data is accessed in a more localized manner, reducing the number of cache misses and increasing the overall performance of the solver 66 . Furthermore, the solver's memory footprint is kept low, which means that it can deal with larger problems or function on computers with limited memory resources.…”

“…The CPU cache efficiency is improved because the data is accessed in a more localized manner, reducing the number of cache misses and increasing the overall performance of the solver. 66 Furthermore, the solver's memory footprint is kept low, which means that it can deal with larger problems or function on computers with limited memory resources.…”

Cache-optimized and low-overhead implementations of additive Schwarz methods for high-order FEM multigrid computations

“…Storing and loading the information for each quadrature point gives a memory-bound algorithm (Kronbichler and Kormann, 2019). In our experiments, we use, on affine meshes, compression that is applicable as J q is the same on several quadrature points and compute the metric terms from a triquadratic representation of a deformed cell geometry (Kronbichler et al, 2022).…”

“…In the context of, for example, discontinuous Galerkin methods, it is possible to store only the index of an unknown of a cell, assuming that all DoFs related to a cell are enumerated contiguously. A similar strategy can be also adopted to standard high-order FEM in the case that DoFs are contiguous on each geometric entity, as proposed by Kronbichler et al (2022). Here, one needs to store only the first index of each entity of a cell (3 d ).…”