Anderson Acceleration for Fixed-Point Iterations

Abstract

Abstract. This paper concerns an acceleration method for fixed-point iterations that originated in work of D. G. Anderson [J. Assoc. Comput. Mach., 12 (1965), pp. 547-560], which we accordingly call Anderson acceleration here. This method has enjoyed considerable success and wide usage in electronic structure computations, where it is known as Anderson mixing; however, it seems to have been untried or underexploited in many other important applications. Moreover, while other acceleration methods have been extensively studied by the mathematics and numerical analysis communities, this method has received relatively little attention from these communities over the years. A recent paper by H. Fang and Y. Saad [Numer. Linear Algebra Appl., 16 (2009), pp. 197-221] has clarified a remarkable relationship of Anderson acceleration to quasi-Newton (secant updating) methods and extended it to define a broader Anderson family of acceleration methods. In this paper, our goals are to shed additional light on Anderson acceleration and to draw further attention to its usefulness as a general tool. We first show that, on linear problems, Anderson acceleration without truncation is "essentially equivalent" in a certain sense to the generalized minimal residual (GMRES) method. We also show that the Type 1 variant in the Fang-Saad Anderson family is similarly essentially equivalent to the Arnoldi (full orthogonalization) method. We then discuss practical considerations for implementing Anderson acceleration and illustrate its performance through numerical experiments involving a variety of applications.

Anderson acceleration of the alternating projections method for computing the nearest correlation matrix

“…Anderson acceleration is related to multisecant methods (extensions of quasi-Newton methods involving multiple secant conditions); in fact, Eyert [16] proves that it is equivalent to the so-called "bad" Broyden's method [11,28], and a similar analysis is done by Fang and Saad [17] and Rohwedder and Schneider [43]. For linear systems, if m k = k for each k then Anderson acceleration is essentially equivalent to the generalized minimal residual (GMRES) method [44], as shown by Potra and Engler [36], Rohwedder and Schneider [43], and Walker and Ni [50]. For nonlinear problems Rohwedder and Schneider [43] show that Anderson acceleration is locally linearly convergent under certain conditions.…”

“…These matrices are related by simple update formulae that can be exploited for an efficient implementation. This variant is given by Fang and Saad [17], Plasse [35], Walker [49], and Walker and Ni [50]. Here, Anderson acceleration is applied to the equivalent problem f (x) = 0, where…”

“…Furthermore, it significantly improved the performance of several domain decomposition methods presented in [50] and has proved to be very efficient on various examples in the above references. Hence Anderson acceleration has great potential for enhancing the convergence of the alternating projections method for the nearest correlation matrix.…”

“…The selected history length m k is usually small. A discussion that puts Anderson acceleration in context with other acceleration methods can be found in [50].…”

Modeling RF waves in spatially dispersive inhomogeneus plasma using an iterative wavelet spectral method

“…Operator splitting is applied to separate dispersive and non-dispersive terms in the wave equation. The dispersive terms are added by means of iteration using Anderson Acceleration scheme [10] and the spatially dispersive response is evaluated using a Morlet wavelet representation of the electric field and dielectric response [11].…”