Adaptive precision in block‐Jacobi preconditioning for iterative sparse linear system solvers

Abstract

Summary We propose an adaptive scheme to reduce communication overhead caused by data movement by selectively storing the diagonal blocks of a block‐Jacobi preconditioner in different precision formats (half, single, or double). This specialized preconditioner can then be combined with any Krylov subspace method for the solution of sparse linear systems to perform all arithmetic in double precision. We assess the effects of the adaptive precision preconditioner on the iteration count and data transfer cost of a preconditioned conjugate gradient solver. A preconditioned conjugate gradient method is, in general, a memory bandwidth‐bound algorithm, and therefore its execution time and energy consumption are largely dominated by the costs of accessing the problem's data in memory. Given this observation, we propose a model that quantifies the time and energy savings of our approach based on the assumption that these two costs depend linearly on the bit length of a floating point number. Furthermore, we use a number of test problems from the SuiteSparse matrix collection to estimate the potential benefits of the adaptive block‐Jacobi preconditioning scheme.

Compression Challenges in Large Scale Partial Differential Equation Solvers

“…Similar results have been obtained for non-overlapping block-Jacobi preconditioners for Krylov methods applied to general sparse systems [37] and for substructuring domain decomposition methods [59].…”

Automatic Exploration of Reduced Floating-Point Representations in Iterative Methods

“…As illustrated in figure 3, the current implementation of VPREC allows modifying the bit length of the exponent r ∈ [1,11] -VPREC operands can be converted to double to use hardware operators to perform VPREC operations with low overhead, including special values (subnormal numbers, NaN and ±∞) related to the user defined format (r, p) -After rounding, converting the result back to double enables graceful degradation if some external libraries are not instrumented.…”

“…We measure a 28% to 67% reduction in the communication volume. The energy gain can be estimated to be linearly related to this volume gain with the simple model proposed in [1].…”

“…-An empirical methodology integrated in Verificarlo [7,14] to automatically lower precision in iterative algorithms while maintaining an user defined accuracy criterion. -A VPREC backend to Verificarlo to emulate variable FP representations with a [1,11]-bits exponent and [0, 52]-bits pseudo-mantissa. -Integration of the temporal dimension in the exploration of FP precision.…”

Power‐aware computing

“…Anzt et al propose an adaptive scheme to reduce communication overhead caused by data movement during the application of block‐Jacobi preconditioners by storing the blocks in different precision formats (half, single, or double). The resulting preconditioner can be combined with any Krylov subspace method for the solution of sparse linear systems, and the authors assess the effects of the adaptive‐precision preconditioner on the iteration count and data transfer cost of a preconditioned conjugate gradient solver.…”