Spectral residual method without gradient information for solving large-scale nonlinear systems of equations

Abstract

Abstract. A fully derivative-free spectral residual method for solving largescale nonlinear systems of equations is presented. It uses in a systematic way the residual vector as a search direction, a spectral steplength that produces a nonmonotone process and a globalization strategy that allows for this nonmonotone behavior. The global convergence analysis of the combined scheme is presented. An extensive set of numerical experiments that indicate that the new combination is competitive and frequently better than well-known Newton-Krylov methods for large-scale problems is also presented.

Residual iterative schemes for large-scale nonsymmetric positive definite linear systems

“…The use of the this steplength is inspired by the success obtained recently for solving nonlinear systems of equations [19,20]. Properties of the spectral step length, α k+1 = (r t k r k )/(r t k Ar k ), for the minimization of convex quadratic functions were established in [22], and further analyzed in [13].…”

“…Obtaining λ k via a backtracking process to force (3) represents a globalization strategy that is inspired by the proposition presented in [20], and requires some given parameters: {η k }, γ , and σ min < σ max . Let us assume that {η k } is a given sequence such that η k > 0 for all k ∈ N (the set of natural numbers) and…”

“…For nonlinear systems, some new iterative schemes have recently been presented that use in a systematic way the residual vectors as search directions [19,20]. These low-memory ideas become effective, and competitive with Newton-Krylov ( [3,9,10,18]) schemes for large-scale nonlinear systems, when the step lengths are chosen in a suitable way.…”

“…In this work we combine and adapt, for linear systems, the ideas introduced in [19,20] for the nonlinear case. To be precise, we present in Section 2 a scheme that takes advantage of the method presented in [19] for choosing the direction, plus or minus the residual, depending on the sign of a Rayleigh quotient closely related to the step length.…”

“…To be precise, we present in Section 2 a scheme that takes advantage of the method presented in [19] for choosing the direction, plus or minus the residual, depending on the sign of a Rayleigh quotient closely related to the step length. It also takes advantage of the new globalization strategy proposed and analyzed in [20] that allows the norm of the residual to decrease non monotonically and still guarantees convergence.…”