On the convergence of Broyden’s method and some accelerated schemes for singular problems

Abstract

We consider Broyden’s method and some accelerated schemes for nonlinear equations having a strongly regular singularity of first order with a one-dimensional nullspace. Our two main results are as follows. First, we show that the use of a preceding Newton-like step ensures convergence for starting points in a starlike domain with density 1. This extends the domain of convergence of these methods significantly. Second, we establish that the matrix updates of Broyden’s method converge q-linearly with the same asymptotic factor as the iterates. This contributes to the long-standing question of whether the Broyden matrices converge by showing that this is indeed the case for the setting at hand. Furthermore, we prove that the Broyden directions violate uniform linear independence, which implies that existing results for convergence of the Broyden matrices cannot be applied. Numerical experiments of high precision confirm the enlarged domain of convergence, the q-linear convergence of the matrix updates and the lack of uniform linear independence. In addition, they suggest that these results can be extended to singularities of higher order and that Broyden’s method can converge r-linearly without converging q-linearly. The underlying code is freely available.

SHINE: SHaring the INverse Estimate from the forward pass for bi-level optimization and implicit models

“…in [12,30,36], it is a strong assumption whose satisfaction in practice is debatable, cf., e.g., [15]. For Broyden's method, ULI is violated in all numerical experiments in [32][33][34], and those works also prove that ULI is necessarily violated in certain settings (but the setting of this work is not covered).…”