Extrapolation Methods for Vector Sequences

Abstract

This is an expository paper that describes and compares five methods for extrapolating to the limit (or anti-limit) of a vector sequence without explicit knowledge of the sequence generator. The methods are the minimal polynomial extrapolation (MPE) studied by Cabay and Jackson, Megina, and Skelboe; the reduced rank extrapolation (RRE) of Eddy (which we show to be equivalent to Megina's version of MPE); the vector and scalar versions of the epsilon algorithm (VEA, SEA) introduced by Wynn and extended by Brezinski and Gekeler; and the topological epsilon algorithm (TEA) of Brezinski. We cover the derivation and error analysis of iterated versions of the algorithms, as applied to both linear and nonlinear problems, and we show why these versions tend to converge quadratically. We also present samples from extensive numerical testing that has led us to the following conclusions: (a) TEA, in spite of its role as a theoretical link between the polynomial-type and the epsilon-type methods, has no practical application; (b) MPE is at least as good as RRE, and VEA at least as good as SEA, in almost all situations; (c) there are circumstances in which either MPE or VEA is superior to the other. Key words, minimal polynomial extrapolation, reduced rank extrapolation, vector epsilon algorithm, scalar epsilon algorithm, topological epsilon algorithm, iterative methods, quadratic convergence AMS(MOS) subject classifications. 65-02, 65B05, 65F10, 65H101. Introduction. The purpose of this paper is to derive, describe, and compare five extrapolation methods for accelerating convergence of vector sequences or transforming divergent vector sequences to convergent ones. These methods have in common the property that they require no explicit knowledge of the "sequence generator" but are computed directly from the terms of the sequence. (In particular, if x+ =F(x), no derivatives of F are computed.) Furthermore, when applied iteratively to a sequence generated nonlinearly, they tend to converge quadratically.The methods studied belong to two families: polynomial methods and epsilon algorithms. The epsilon algorithms are old enough to be called "classical" in this field, dating to the mid-1950's for scalar sequences and to the early 1960's for vector sequences. We consider the scalar epsilon algorithm (SEA) and vector epsilon algorithm (VEA) introduced by Wynn [36] and the more recent topological epsilon algorithm (TEA) of Brezinski [8]. Polynomial extrapolation methods that are linear in the terms of the sequence, such as Chebyshev acceleration, have at least as long a history, but we consider only methods that are nonlinear, in that the coefficients of the extrapolating polynomials are functions of the terms of the sequence. The minimal polynomial extrapolation (MPE) and reduced rank extrapolation (RRE) are adapted from the works of Cabay and Jackson [12], Eddy [13], [14], Medina [26], and Skelboe [31].All five methods are based on the idea of exact solution for a fixed point in the case of a linear generator F, but without the equi...

Topologically Constrained Isometric Embedding

“…As shown by [24], the coef cients of this polynomial can be computed from a linear equation in {u (j) }. but does so with smaller matrices U, V, of size N × k.…”

“…An overview of these algorithms is given here, as these algorithms were previously described in details [22], [23], [24]. In describing these algorithms we use the following notation: Let {x i } i=0 be a series of iteration vectors resulting from an iterative process.…”

Modelling and Discretization of Circuit Problems

“…In the past this approach has been applied by [224] (even already for circuit problems) and [184]. In [226] extrapolation techniques were generalized to sequences of vectors and has resulted in methods like Minimal Polynomial Extrapolation (MPE) and Reduced Rank Extrapolation (RRE). It is worth noting that all these methods can be implemented very elegantly within existing circuit simulators.…”

“…A well-known acceleration method is minimal polynomial extrapolation (MPE). Rather than describing MPE here in detail, the reader is referred to [226]. For results with this approach we refer to [122][123][124].…”

An Accelerated Poincaré-Map Method for Finding the PSS of Autonomous Oscillators

“…A well-known acceleration method is minimal polynomial extrapolation (MPE). Rather than describing MPE here in detail, the reader is referred to [5]. The MPE-accelerated Poincare-map method has also been tested on Colpitt's Oscillator.…”