Performance of Low Synchronization Orthogonalization Methods in Anderson Accelerated Fixed Point Solvers

Lockhart, Shelby; Gardner, David J.; Woodward, Carol S.; Thomas, Stephen J.; Olson, Luke N.

doi:10.1137/1.9781611977141.5

Cited by 2 publications

(2 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Krylov subspace methods for solving linear systems are often required for extreme-scale physics simulations on parallel machines with manycore accelerators. Their strong-scaling is limited by the number and frequency of global reductions in the form of MPI AllReduce operations and these communication patterns are expensive [13]. Lowsynchronization algorithms are based on the ideas of Ruhe [5], and are designed such that they require only one reduction per iteration to normalize each vector and apply projections.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Iterated Gauss-Seidel GMRES

Thomas¹,

Carson²,

Rozložník³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

The GMRES algorithm of Saad and Schultz (1986) for nonsymmetric linear systems relies on the Arnoldi expansion of the Krylov basis. The algorithm computes the QR factorization of the matrix B = [r 0 , AV k ] at each iteration. Despite an O(ε)κ(B) loss of orthogonality, the modified Gram-Schmidt (MGS) formulation was shown to be backward stable in the seminal papers by Paige, et al. (2006) and Paige and Strakoš (2002). Classical Gram-Schmidt (CGS) exhibits an O(ε)κ 2 (B) loss of orthogonality, whereas DCGS-2 (CGS with delayed reorthogonalization) reduces this to O(ε) in practice (without a formal proof). We present a post-modern (viz. not classical) GMRES algorithm based on Ruhe (1983) and the low-synch algorithms of Świrydowicz et al (2020) that achieves O(ε) Av k 2 /h k+1,k loss of orthogonality. By projecting the vector Av k , with Gauss-Seidel relaxation, onto the orthogonal complement of the space spanned by the computed Krylov vectors V k where V T k V k = I + L k + L T k , we can further demonstrate that the loss of orthogonality is at most O(ε)κ(B). For a broad class of matrices, unlike MGS-GMRES, significant loss of orthogonality does not occur and the relative residual no longer stagnates for highly non-normal systems. The Krylov vectors remain linearly independent and the smallest singular value of V k is not far from one. We also demonstrate that Henrici's departure from normality of the lower triangular matrixk is an appropriate quantity for detecting the loss of orthogonality. Our new algorithm results in an almost symmetric correction matrix T k .

show abstract

Section: Introductionmentioning

confidence: 99%

“…The low-synch modified Gram-Schmidt and GMRES algorithms described in Świrydowicz et al [3] improve parallel strong-scaling by employing one global reduction for each iteration, (see Lockhart et al [13]). A review of compact W Y Gram Schmidt algorithms and their computational costs is given in [4].…”

Section: Introductionmentioning

confidence: 99%