Convergence Analysis of Iterative Solvers in Inexact Rayleigh Quotient Iteration

Abstract

Abstract. We present a detailed convergence analysis of preconditioned MINRES for approximately solving the linear systems that arise when Rayleigh Quotient Iteration is used to compute the lowest eigenpair of a symmetric positive definite matrix. We provide insight into the "slow start" of MINRES iteration in both a qualitative and quantitative way, and show that the convergence of MINRES mainly depends on how quickly the unique negative eigenvalue of the preconditioned shifted coefficient matrix is approximated by its corresponding harmonic Ritz value. By exploring when the negative Ritz value appears in MINRES iteration, we obtain a better understanding of the limitation of preconditioned MINRES in this context and the virtue of a new type of preconditioner with "tuning". Comparison of MINRES with SYMMLQ in this context is also given. Finally we show that tuning based on a rank-2 modification can be applied with little additional cost to guarantee positive definiteness of the tuned preconditioner.Key words. Rayleigh Quotient Iteration, harmonic Ritz value, MINRES, tuned preconditioner 1. Introduction. There has been considerable interest in recent years in developing and analyzing eigensolvers with inner-outer structure for computing eigenvalues of matrices closest to some specified value. These algorithms usually involve at each step (outer iteration) a shift-invert matrix-vector product implemented by solving the shifted linear system iteratively (inner iteration). The use of inner iteration becomes mandatory if the matrices are too large for factorization-based exact shift-invert matrix-vector products to be practical. Inexact inverse iteration is the most simple algorithm of this type and the best understood one. Early papers on the convergence of inexact inverse iteration with fixed shift include [8] and [9], where the main concern is to choose a decreasing sequence of stopping tolerances for inner solvers to maintain linear convergence of the outer iteration. Analysis of inexact Rayleigh Quotient Iteration (RQI) in [13] and [18] shows how the inexactness of the inner solve can affect the convergence of the outer iteration. More recent work focuses on improving the convergence of inner iterations as well as the relation between the inner and outer iterations. Reference [16] introduces some new perspectives on preconditioning in this setting, namely, that faster convergence of inner iterations can be obtained by modifying the right hand side of the preconditioned linear system. Refined analysis of this approach in [1], [2] and [5] shows how different formulations of the linear system, with variable shift and different inner stopping criteria, can affect the convergence of the inner and outer iterations. An alternative preconditioning approach called "tuning" is analyzed in [6] for non-symmetric eigenvalue problems and in [7] for symmetric problems. A preconditioner with tuning is a low rank modification of an ordinary preconditioner. Tuning forces the preconditioning operator to behave in the same way as...

Efficient Preconditioned Inner Solves For Inexact Rayleigh Quotient Iteration And Their Connections To The Single-Vector Jacobi–Davidson Method

“…Note that Q constructed in (3.9a) and (3.9b) is Hermitian if Q and B are both Hermitian; in addition, it is shown in [31] that Q defined in (3.9b) is positive definite if B and Q are positive definite. Therefore it is advisable to construct a tuned preconditioner by (3.9b) for the inner solves for Hermitian problems.…”

“…The eigenvector approximation continues to improve as the inner iteration proceeds, and the difficulty arising in the solution of (3.8) with (untuned) Q is thus resolved (see [31] for a detailed discussion of the virtue of the tuned preconditioner).…”

“…For standard Hermitian problems, Simoncini and Eldén [24] and Xue and Elman [31] provide evidence that t m ¼ gtan ∠ ðy m ; v 1 Þ can be much larger than t 0 ¼ gtan ∠ ðx; v 1 Þ for a large number of inner iterations, in which ∠ðy m ; v 1 Þ decreases very slowly with m, until m becomes large enough. This undesirable behavior, in fact, can also be observed for standard non-Hermitian and generalized problems.…”

“…As a result, the linear residual norm decreases more quickly in the asymptotic convergence phase for the untuned preconditioned solve. On the other hand, it is suggested in [31] that for IRQI for standard Hermitian problems, the asymptotic convergence rate of solving ðA − σI Þ ¼ x by MINRES with a tuned preconditioner may be a reasonable indication of the rate at which the eigenvalue residual norm of inner iterates decreases. The correlation of the two rates may also hold for non-Hermitian problems, and thus it provides some explanation into the advantage of GMRES with untuned preconditioner in the later stage of inner iterations.…”

“…To evaluate the efficiency of inner solves of IRQI, one is most concerned about how quickly significant improvement of eigenvector approximation can be achieved as the inner iteration proceeds. In terms of preconditioners for the inner solve, it is shown in [9], [10], [13], [31] that a regular preconditioner Q used in the usual setting of solving linear systems is less competitive than a tuned variant Q that satisfies Qx ¼ Ax or Qx ¼ Bx. The major reason is that the initial iterate of the preconditioned inner solve with tuning is y 1 ¼ Q −1 Bx ¼ x up to a scaling factor (assuming that Qx ¼ Bx), i.e., the best eigenvector approximation currently available; in addition, x is kept in the subspace of approximate solutions throughout the inner iteration process.…”