Convergence Theorems for Least-Change Secant Update Methods

Abstract

Abstract. The purpose of this paper is to present a convergence analysis of least change secant methods in which part of the derivative matrix being approximated is computed by other means. many readers might feel that it is easier to understand. The theorems here readily imply local and q-superlinear convergence of all the standard methods in addition to proving these results for the first time for the sparse symmetric method of Marwil and Toint and the nonlinear least-squares method of DennisGay-Welsch.

The column-updating method for solving nonlinear equations in Hilbert space

“…The existence of a local convergence resuit for CUM without restarts (q = oo ) may be conjectured. This resuit, as well as the corresponding superlinear convergence without restarts, is not easy to obtain, since gênerai convergence théories [9,18] are not applicable. We think that this conjecture deserves future research.…”

“…However, let us observe that, if B k + l is chosen according to (2.5) and 0 k = 1, the sécant équation (2.7) holds, independently of the définition of s k . It is easy to see that, if ^ = -À k Bj~lF(x k ) 9 we have…”

“…This condition is not necessary for many methods for solving nonlinear simultaneous équations. (See [2,3,4,6,7,9,18,19]). …”

Local convergence theory of inexact Newton methods based on structured least change updates

“…Dennis, Gay, and Welsch [8] proposed to replace it by a weighted secant approximation, introducing a DFP type update. Later, Dennis and Walker [ 14] proposed a BFGS type update formula for the difference between J(x)~ and [R'(x) R(x)]~ . Both methods are particular cases of the Model Algorithm 2.1.…”

Direct secant updates of matrix factorizations

“…Recent work, [8], [14], has convinced us that these requirements are necessary for convergence and numerical stability even though we cannot give a rigorous proof of this conjecture. We have no doubt that the situation will become clarified in time, but we feel that it is silly to delay publication any longer of a potentially useful numerical method just because the analysis of the method is not provably sharp.…”