Analyzing and improving maximal attainable accuracy in the communication hiding pipelined BiCGStab method

Abstract

Pipelined Krylov subspace methods avoid communication latency by reducing the number of global synchronization bottlenecks and by hiding global communication behind useful computational work. In exact arithmetic pipelined Krylov subspace algorithms are equivalent to classic Krylov subspace methods and generate identical series of iterates. However, as a consequence of the reformulation of the algorithm to improve parallelism, pipelined methods may suffer from severely reduced attainable accuracy in a practical finite precision setting. This work presents a numerical stability analysis that describes and quantifies the impact of local rounding error propagation on the maximal attainable accuracy of the multi-term recurrences in the preconditioned pipelined BiCGStab method. Theoretical expressions for the gaps between the true and computed residual as well as other auxiliary variables used in the algorithm are derived, and the elementary dependencies between the gaps on the various recursively computed vector variables are analyzed. The norms of the corresponding propagation matrices and vectors provide insights in the possible amplification of local rounding errors throughout the algorithm. Stability of the pipelined BiCGStab method is compared numerically to that of pipelined CG on a symmetric benchmark problem. Furthermore, numerical evidence supporting the effectiveness of employing a residual replacement type strategy to improve the maximal attainable accuracy for the pipelined BiCGStab method is provided.

Recent Developments in Iterative Methods for Reducing Synchronization

“…In finite precision arithmetic, Cools et al [19] discussed the the effect of local rounding error propagation on the maximal attainable accuracy of the pipelined CG method and compared it with the classical CG and Chronopoulos-Gear CG [16]. In a later paper, Cools [17] gave a similar discussion for the pipelined BICGSTAB method. Carson et al [12] discussed the stability issues for synchronization-reducing algorithms and presented a methodology for the theoretical analysis of some CG variants.…”

Numerically Stable Recurrence Relations for the Communication Hiding Pipelined Conjugate Gradient Method

“…All Conjugate Gradient variants listed in Table 1 compute a single SPMV in each iteration. However, as indicated by if i ≥ l then # Finalize dot-products (g j,i−l+1 ) 8:…”

“…Although Alg. 1, line 18 indicates that in each iteration i ≥ (2l + 1) a total of (2l + 1) dot products need to be computed, the number of dot product computations can be limited to (l + 1) by exploiting the symmetry of the matrix G i+1 , see expression (8). Since g j,i+1 = g i−l+1,j+l for j ≤ i + 1, only the dot products (z j , z i+1 ) for j = i − l + 2, .…”

“…However, the algorithmic reformulations that allows for this efficiency increase come at the cost of reduced numerical stability [5], [19], which presently is one of the main drawbacks of pipelined (as well as other communication reducing) methods. Research on analyzing and improving the numerical stability of pipelined Krylov subspace methods, which is essential both for a proper understanding and the practical usability of the methods, has recently been performed by the authors [8], [10] and others [3], [5].…”

Numerical Modelling of TEM Fields by the Edge-Based Finite Element Method Using Tetrahedral Element for Central Rectangular Loops

“…Iterative methods are computationally more attractive than the direct methods particularly for large and sparse systems (Freund et al 1992, Dreyer 2009. There are a wide variety of iterative techniques, such as Conjugate Gradient (CG), minimum residual, generalized minimum residual, Bi-conjugate gradient, quasi minimal residual, conjugate gradient squared, Bi-conjugate Gradient Stabilized (BiCGStab) and Chebyshev iterations (Cools 2019). We used the Corrected Bi Conjugate Gradient Stabilized Method (CBiCGStab) for the solutions of equations with unsymmetric coefficient matrices.…”