GMRES with Deflated Restarting

Abstract

A modification is given of the GMRES iterative method for nonsymmetric systems of linear equations. The new method deflates eigenvalues using Wu and Simon's thick restarting approach [SIAM J. Matrix Anal. Appl., 22 (2000), pp. 602-616]. It has the efficiency of implicit restarting but is simpler and does not have the same numerical concerns. The deflation of small eigenvalues can greatly improve the convergence of restarted GMRES. Also, it is demonstrated that using harmonic Ritz vectors is important because then the whole subspace is a Krylov subspace that contains certain important smaller subspaces. [38,35] for solving large systems of linear equations. It deflates eigenvalues using the thick restarting technique given by Wu and Simon in [48] for the Lanczos eigenvalue method. Deflation can significantly improve the convergence of restarted GMRES, and it helps robustness by allowing the solution of many tough problems that have small eigenvalues. Introduction. We develop a new version of GMRESThe new method, called GMRES with deflated restarting or GMRES-DR, is mathematically equivalent to the method GMRES with eigenvectors [23] and to two methods in [25], including implicitly restarted GMRES. Thus this paper could be titled, "Yet another equivalent deflated GMRES method." However, the new approach has the efficiency of implicit restarting but is simpler and does not have the numerical concerns. Thus it should be a useful improvement.In the next section, we discuss deflation for Krylov methods, including some previous approaches and results. Section 3 presents the new method, including a variant that removes approximate eigenvectors from the Krylov subspace. Then, in section 4, numerical examples are given that illustrate some points, including the importance of having Krylov subspaces with approximate eigenvectors as starting vectors.

A Flexible Generalized Conjugate Residual Method with Inner Orthogonalization and Deflated Restarting

“…A number of techniques have been proposed to compute spectral information at a restart and use this information to improve the convergence rate of the Krylov subspace methods; see, e.g., [16,17,18,24]. These techniques have been exclusively developed in the case of a fixed preconditioner.…”

“…A full subspace of dimension k, k < m (and not only the approximate solution with minimum residual norm) is now retained at the restart and the success of this approach has been demonstrated on many academic examples [16]. Approximations of eigenvalues of smallest magnitude are obtained by computing harmonic Ritz pairs of A with respect to a certain subspace [18]. We present here a definition of a harmonic Ritz pair equivalent to the one introduced in [21,30]; it will be of key importance when defining appropriate deflation strategies.…”

“…• As pointed out by Morgan [18] and Parks et al [22] we might have to adjust k during the algorithm to include both the real and imaginary parts of complex eigenvectors.…”

“…Deflated methods compute spectral information at a restart and use this information to improve the convergence of the subspace method. One of the most recent procedure based on a deflation approach is GMRES with deflated restarting (GMRES-DR) [18]. This method reduces to restarted GMRES when no deflation is applied, but may provide a much faster convergence than restarted GMRES for well chosen deflation spaces as described in [18].…”

Adaptively Accelerated Gmres Fast Fourier Transform Method for Electromagnetic Scattering

“…A disadvantage of this approach is that the convergence behavior in many situations seems to depend quite critically on the value of m. Even in situations in which satisfactory convergence takes place, the convergence is less than optimal, since the history is thrown away so that potential superlinear convergence behavior is inhibited [10]. There are many acceleration techniques that attempt to mimic the convergence of full GMRES more closely, or to accelerate the convergence of the regular GMRES by retaining some historical information at the time of restart [11][12][13][14][15]. Deflation methods are a main class of acceleration techniques for GMRES.…”