Abstract. The generalized minimum residual method (GMRES) [Y. Saad and M. Schultz, SIAM J. Sci. Statist. Comput., 7 (1986), pp. 856-869] for solving linear systems Ax = b is implemented as a sequence of least squares problems involving Krylov subspaces of increasing dimensions. The most usual implementation is modified Gram-Schmidt GMRES (MGS-GMRES). Here we show that MGS-GMRES is backward stable. The result depends on a more general result on the backward stability of a variant of the MGS algorithm applied to solving a linear least squares problem, and uses other new results on MGS and its loss of orthogonality, together with an important but neglected condition number, and a relation between residual norms and certain singular values.Key words. rounding error analysis, backward stability, linear equations, condition numbers, large sparse matrices, iterative solution, Krylov subspace methods, Arnoldi method, generalized minimum residual method, modified Gram-Schmidt, QR factorization, loss of orthogonality, least squares, singular values AMS subject classifications. 65F10, 65F20, 65F25, 65F35, 65F50, 65G50, 15A12, 15A42 DOI. 10.1137/050630416 1. Introduction. Consider a system of linear algebraic equations Ax = b, where A is a given n × n (unsymmetric) nonsingular matrix and b a nonzero n-dimensional vector. Given an initial approximation x 0 , one approach to finding x is to first compute the initial residual r 0 = b − Ax 0 . Using this, derive a sequence of Krylov subspaces. . , in some way, and look for approximate solutions x k ∈ x 0 + K k (A, r 0 ) . Various principles are used for constructing x k , which determine various Krylov subspace methods for solving Ax = b. Similarly, Krylov subspaces for A can be used to obtain eigenvalue approximations or to solve other problems involving A.Krylov subspace methods are useful for solving problems involving very large sparse matrices, since these methods use these matrices only for multiplying vectors, and the resulting Krylov subspaces frequently exhibit good approximation properties. The Arnoldi method [2] is a Krylov subspace method designed for solving the eigenproblem of unsymmetric matrices. The generalized minimum residual method (GMRES) [20] uses the Arnoldi iteration and adapts it for solving the linear system Ax = b. GMRES can be computationally more expensive per step than some other methods; see, for example, Bi-CGSTAB [24] and QMR [9] for unsymmetric A, and LSQR [16] for unsymmetric or rectangular A. However, GMRES is widely used for solving linear systems arising from discretization of partial differential equations, and

This paper provides two results on the numerical behavior of the classical Gram-Schmidt algorithm. The first result states that, provided the normal equations associated with the initial vectors are numerically nonsingular, the loss of orthogonality of the vectors computed by the classical Gram-Schmidt algorithm depends quadratically on the condition number of the initial vectors. The second result states that, provided the initial set of vectors has numerical full rank, two

In this paper, we study numerical behavior of several computational variants of the Gram-Schmidt orthogonalization process. We focus on the orthogonality of computed vectors which may be significantly lost in the classical or modified Gram-Schmidt algorithm, while the GramSchmidt algorithm with reorthogonalization has been shown to compute vectors which are orthogonal to machine precision level. The implications for practical implementation and its impact on the efficiency in the parallel computer environment are considered. ~)

No abstract

No abstract