Anderson Acceleration for Fixed-Point Iterations

Abstract

This paper concerns an acceleration method for fixed-point iterations that originated in work of D. G. Anderson [J. Assoc. Comput. Mach., 12 (1965), pp. 547-560], which we accordingly call Anderson acceleration here. This method has enjoyed considerable success and wide usage in electronic structure computations, where it is known as Anderson mixing; however, it seems to have been untried or underexploited in many other important applications. Moreover, while other acceleration methods have been extensively studied by the mathematics and numerical analysis communities, this method has received relatively little attention from these communities over the years. A recent paper by H. Fang and Y. Saad [Numer. Linear Algebra Appl., 16 (2009), pp. 197-221] has clarified a remarkable relationship of Anderson acceleration to quasi-Newton (secant updating) methods and extended it to define a broader Anderson family of acceleration methods. In this paper, our goals are to shed additional light on Anderson acceleration and to draw further attention to its usefulness as a general tool. We first show that, on linear problems, Anderson acceleration without truncation is "essentially equivalent" in a certain sense to the generalized minimal residual (GMRES) method. We also show that the Type 1 variant in the Fang-Saad Anderson family is similarly essentially equivalent to the Arnoldi (full orthogonalization) method. We then discuss practical considerations for implementing Anderson acceleration and illustrate its performance through numerical experiments involving a variety of applications.

On Quasi‐Newton methods in fast Fourier transform‐based micromechanics

“…One such algorithm is Anderson acceleration which was recently applied by Shantraj et al and Chen et al in the context of FFT‐based micromechanics. A general discussion of the scheme and its implementation is found, for example, in Walker and Ni or Kelley . Eyert and Fang and Saad pointed out the relation of Anderson acceleration to Quasi‐Newton schemes and identified it as a generalized form of Broyden's second method.…”

Convergence analysis of Anderson‐type acceleration of Richardson's iteration

“…Similarly to other works on AR, it is possible to identify a connection between AAR and GMRES . Let us assume again that m = k at first.…”

“…Anderson mixing is thus well suited to potentially accelerate Richardson schemes. Contributions in the mathematical literature in this respect have resulted in an algorithm called the Anderson–Richardson method (AR for short) . Studies about AR have highlighted similarities between this method and GMRES.…”

“…In the present work, we study the mathematical properties of AAR in comparison with more standard techniques such as full GMRES, truncated GMRES, restarted GMRES, and Anderson–Richardson . All the convergence analyses and comparisons between algorithms conducted in this work assume exact arithmetic.…”

Gradient‐based nodal limiters for artificial diffusion operators in finite element schemes for transport equations

“…Because of the dependence of the correction factors α i j and β i j on the unknown solution, the algebraic system is nonlinear. It can be solved using the fixed‐point iteration In applications to anisotropic diffusion problems, the rates of convergence to steady‐state solutions can be greatly improved using Anderson acceleration for fixed‐point iterations .…”