Modified Gram-Schmidt (MGS), Least Squares, and Backward Stability of MGS-GMRES

Paige, Christopher C.; Rozložńık, Miroslav; Strakoš, Zdeněk

doi:10.1137/050630416

Cited by 100 publications

(120 citation statements)

References 22 publications

Supporting

Mentioning

116

Contrasting

Unclassified

Order By: Relevance

“…For any backward stable solver (such as LU factorization with appropriate pivoting for stability, or GMRES with Householder orthogonalization [11] or modified Gram-Schmidt orthogonalization [28]) we know that the backward error r i 2 /( A 2 x i 2 ) of the computed solution x i to Ax = b will be small, yet the forward error may be large. For the refinement, the initial backward error will be small and the same will be true for each iterate x i , as refinement does not degrade the backward error.…”

mentioning

confidence: 99%

See 1 more Smart Citation

A New Analysis of Iterative Refinement and Its Application to Accurate Solution of Ill-Conditioned Sparse Linear Systems

Carson¹,

Higham

2017

SIAM J. Sci. Comput.

126

View full text Add to dashboard Cite

Abstract. Iterative refinement is a long-standing technique for improving the accuracy of a computed solution to a nonsingular linear system Ax = b obtained via LU factorization. It makes use of residuals computed in extra precision, typically at twice the working precision, and existing results guarantee convergence if the matrix A has condition number safely less than the reciprocal of the unit roundoff, u. We identify a mechanism that allows iterative refinement to produce solutions with normwise relative error of order u to systems with condition numbers of order u −1 or larger, provided that the update equation is solved with a relative error sufficiently less than 1. A new rounding error analysis is given, and its implications are analyzed. Building on the analysis, we develop a GMRES (generalized minimal residual)-based iterative refinement method (GMRES-IR) that makes use of the computed LU factors as preconditioners. GMRES-IR exploits the fact that even if A is extremely ill conditioned the LU factors contain enough information that preconditioning can greatly reduce the condition number of A. Our rounding error analysis and numerical experiments show that GMRES-IR can succeed where standard refinement fails, and that it can provide accurate solutions to systems with condition numbers of order u −1 and greater. Indeed, in our experiments with such matrices-both random and from the University of Florida Sparse Matrix Collection-GMRES-IR yields a normwise relative error of order u in at most three steps in every case.

show abstract

mentioning

confidence: 99%

“…Then we use the analysis of [28] to show that our GMRES variant provides a backward stable solution to Ad i = s i . These three results allow us to conclude that Ad i = s i can be solved with some degree of relative accuracy; that is, (2.1) is satisfied.…”

mentioning

confidence: 99%

A New Analysis of Iterative Refinement and Its Application to Accurate Solution of Ill-Conditioned Sparse Linear Systems

Carson¹,

Higham

2017

SIAM J. Sci. Comput.

126

View full text Add to dashboard Cite

show abstract

“…Há dois resultados em [91] e [96] onde são caracterizadas a estabilidade em relação ao erro inverso das implementações do GMRES usando as reflexões de Householder e o método modificado de GramSchmidt no processo de Arnoldi. Esses resultados asseguram que pequenas modificações nos dados tratados não irão acarretar grande problemas à solução do problema, uma vez que se estará resolvendo exatamente um problema próximo.…”

Section: 2)unclassified

“…Como falamos anteriormente, para matrizes não simétricas o GMRES [102] é frequentemente escolhido devido a sua robustez. Como apontamos na seção 2.3, há trabalhos sobre o tema em [91] e [96] onde são caracterizadas a estabilidade em relação ao erro inverso das implementações do GMRES, usando as reflexões de Householder e o método modificado de Gram-Schmidt no processo de Arnoldi. Uma outra razão da popularidade do método é que a norma euclidiana do resíduo não cresce (em geral decresce) durante o avanço das iterações.…”

Section: Estudo De Caso: Gmres Flexível Com Recomeço Deflacionadounclassified

Avanços em Métodos de Krylov para Solução de Sistemas Lineares de Grande Porte

2009

Notas Em Matemática Aplicada

View full text Add to dashboard Cite

“…Notice that the given y solves a problem within the range of uncertainty in the data (2.13) if and only if η 2,F ≤ 1, so that if η 2,F ≤ 1 we can conclude that the given y is an acceptable solution to the compatible system Ax = b, and this can be used as a stopping criterion for iterative methods such as MGS-GMRES (see [15]). …”

Section: Minimal Backward Errors and Acceptablementioning

confidence: 99%

Characterizing Matrices That Are Consistent with Given Solutions

Chang

Paige²,

Titley-Péloquin³

2009

SIAM J. Matrix Anal. & Appl.

View full text Add to dashboard Cite

Abstract. For given vectors b ∈ C m and y ∈ C n we describe a unitary transformation approach to deriving the set F of all matrices F ∈ C m×n such that y is an exact solution to the compatible system F y = b. This is used for deriving minimal backward errors E and f such that (A+E)y = b+f when possibly noisy data A ∈ C m×n and b ∈ C m are given, and the aim is to decide if y is a satisfactory approximate solution to Ax = b. The approach might be different, but the above results are not new. However we also prove the apparently new result that two well known approaches to making this decision are theoretically equivalent, and discuss how such knowledge can be used in designing effective stopping criteria for iterative solution techniques. All these ideas generalize to the following formulations. We extend our constructive approach to derive a superset F ST LS+ of the set F ST LS of all matrices F ∈ C m×n such that y is a scaled total least squares solution to F y ≈ b. This is a new general result that specializes in two important ways. The ordinary least squares problem is an extreme case of the scaled total least squares problem, and we use our result to obtain the set F LS of all matrices F ∈ C m×n such that y is an exact least squares solution to F y ≈ b. This complements the original less-constructive derivation of Waldén, Karlson and Sun [Numerical Linear Algebra with Applications, 2:271-286 (1995)]. We do the equivalent for the data least squares problem-the other extreme case of the scaled total least squares problem. Not only can the results be used as indicated above for the compatible case, but the constructive technique we use could also be applicable to other backward problems-such as those for under-determined systems, the singular value decomposition, and the eigenproblem.Key words. matrix characterization, approximate solutions, iterative methods, linear algebraic equations, least squares, data least squares, total least squares, scaled total least squares, backward errors, stopping criteria.AMS subject classifications. 15A06, 15A29, 65F05, 65F10, 65F20, 65F25, 65G99. DOI. (Digital Object Identifier)1. Introduction. We will study a class of 'backward' problems for linear systems F y ≈ b. Specifically, given two vectors y and b we want to find the sets of all matrices F such that y is the exact solution (i.e., F y = b), the least squares (LS) solution, the data least squares (DLS) solution, and the scaled total least square (STLS) solution. We will propose a unified unitary transformation approach to handling these problems.Some of these problems have been investigated before. The result for the compatible case is well-known and the result for the least squares case was obtained elegantly by Waldén, Karlson and Sun in [31]. But while [31] presents, then proves, the least squares result, our approach shows how to derive a more general result in a fairly simple way, and we suspect that this constructive approach is not only easier to comprehend for non-mathematicians, but perhaps easier to apply to ot...

show abstract

Modified Gram-Schmidt (MGS), Least Squares, and Backward Stability of MGS-GMRES

Cited by 100 publications

References 22 publications

A New Analysis of Iterative Refinement and Its Application to Accurate Solution of Ill-Conditioned Sparse Linear Systems

A New Analysis of Iterative Refinement and Its Application to Accurate Solution of Ill-Conditioned Sparse Linear Systems

Avanços em Métodos de Krylov para Solução de Sistemas Lineares de Grande Porte

Characterizing Matrices That Are Consistent with Given Solutions

Contact Info

Product

Resources

About