Benchmarking results for the Newton–Anderson method

Abstract

This paper primarily presents numerical results for the Anderson accelerated Newton method on a set of benchmark problems. The results demonstrate superlinear convergence to solutions of both degenerate and nondegenerate problems. The convergence for nondegenerate problems is also justified theoretically. For degenerate problems, those whose Jacobians are singular at a solution, the domain of convergence is studied. It is observed in that setting that Newton-Anderson has a domain of convergence similar to Newton, but it may be attracted to a different solution than Newton if the problems are slightly perturbed.

Newton-Anderson at Singular Points

“…The constants C 1 and C 2 are independent of k and depend on f . The bound in (11) resembles the result of Lemma 1 in [20] for depth m = 1. There, the Jacobian is assumed to be nonsingular at the solution x * .…”

“…Here, we will focus on the analysis of Anderson acceleration applied to Newton's method for a problem of the form f (x) = 0, when the derivative f is singular at a root x * . Rapid convergence of the accelerated scheme in comparison with standard Newton has been demonstrated numerically in this singular case [20], where it is also observed that it is generally both sufficient and preferable to set the algorithmic depth to m = 1. In the remainder, we will consider Anderson acceleration with depth m = 1 applied to Newton iterations, which we will refer to simply as Newton-Anderson.…”

Fast convergence to higher multiplicity zeros

“…One of the motivations for looking at the scalar version of Algorithm 1 applied to the Newton method was to understand the method in this simpler setting to gain insight into its use in a more general setting. As a result of this investigation, and as demonstrated in [10], it was found for f : R n → R n , the Newton-Anderson method can provide superlinear convergence to solutions of degenerate problems, those whose Jacobians are singular at a solution (and for which Newton converges only linearly), as well as nondegenerate problems (where Newton converges quadratically). This paper focuses on f : R → R, and provides analytical and numerical results to characterize the scalar case.…”

On the convergence of Broyden's method and some accelerated schemes for singular problems

“…[Gay77, Gay79a, DK80, Gri80b] for Newton's method, [DK83] for the chord method, [KX93] for inexact Newton methods, [DK82, KS83, Kel86, SY05] for accelerated schemes, [Ral66, Red79, DKK83] for Newton's method and accelerated schemes, and the survey article [Gri85]. More recent works are [OW09], where the smoothness assumptions on the Jacobian are weakened to strong semismoothness, [IKS18b,IKS18a,FIS19], where various Newton-type methods are considered, [FIS21b,FIS21a], where line-search strategies and acceleration are investigated, and the numerical study [PS20] of the Newton-Anderson method that includes results for singular problems. In [GO83, IU15] Newton's method is analyzed for singular optimization problems.…”