Iterated Gauss-Seidel GMRES

Abstract

The GMRES algorithm of Saad and Schultz (1986) for nonsymmetric linear systems relies on the Arnoldi expansion of the Krylov basis. The algorithm computes the QR factorization of the matrix B = [r 0 , AV k ] at each iteration. Despite an O(ε)κ(B) loss of orthogonality, the modified Gram-Schmidt (MGS) formulation was shown to be backward stable in the seminal papers by Paige, et al. (2006) and Paige and Strakoš (2002). Classical Gram-Schmidt (CGS) exhibits an O(ε)κ 2 (B) loss of orthogonality, whereas DCGS-2 (CGS with delayed reorthogonalization) reduces this to O(ε) in practice (without a formal proof). We present a post-modern (viz. not classical) GMRES algorithm based on Ruhe (1983) and the low-synch algorithms of Świrydowicz et al (2020) that achieves O(ε) Av k 2 /h k+1,k loss of orthogonality. By projecting the vector Av k , with Gauss-Seidel relaxation, onto the orthogonal complement of the space spanned by the computed Krylov vectors V k where V T k V k = I + L k + L T k , we can further demonstrate that the loss of orthogonality is at most O(ε)κ(B). For a broad class of matrices, unlike MGS-GMRES, significant loss of orthogonality does not occur and the relative residual no longer stagnates for highly non-normal systems. The Krylov vectors remain linearly independent and the smallest singular value of V k is not far from one. We also demonstrate that Henrici's departure from normality of the lower triangular matrixk is an appropriate quantity for detecting the loss of orthogonality. Our new algorithm results in an almost symmetric correction matrix T k .