On the Power of Adaption

Abstract

Optimal error bounds for adaptive and nonadaptive numerical methods are compared. Since the class of adaptive methods is much larger, a well-chosen adaptive method might seem to be better than any nonadaptive method. Nevertheless there are several results saying that under natural assumptions adaptive methods are not better than nonadaptive ones. There are also other results, however, saying that adaptive methods can be significantly better than nonadaptive ones as well as bounds on how much better they can be. It turns out that the answer to the ''adaption problem'' depends very much on what is known a priori about the problem in question; even a seemingly small change of the assumptions can lead to a different answer. © 1996 Academic Press, Inc. THE ADAPTION PROBLEMOne of the more controversial issues in numerical analysis concerns adaptive algorithms. The use of such algorithms is widespread and many people believe that well-chosen adaptive algorithms are much better than nonadaptive methods in most situations. Such a belief is usually based on numerical experimentation. In this paper we survey what is known theoretically regarding the power of adaption. We will present some results which state that under natural assumptions adaptive methods are not better than nonadaptive ones. There are also other results, however, saying that adaptive methods can be significantly superior to nonadaptive ones. As we will see, the power of adaption is critically dependent on our a priori knowledge concerning the problem being studied; even a seemingly small change in the assumptions can lead to a different answer. ERICH NOVAKLet us begin with some well-known examples. The bisection method and the Newton method for zero finding of a function are adaptive, since they compute a sequence (x n ) n of knots that depends on the function. The Gauss formula for numerical integration is nonadaptive since its knots and weights do not depend on the function.A nonadaptive method provides an immediate decomposition for parallel computation. If adaptive information is superior to nonadaptive information, then an analysis of the tradeoff between using adaptive or nonadaptive information on a parallel computer should be carried out.To formulate the adaption problem precisely, we need some definitions and notations. Many problems of numerical analysis can be described as computing an approximation of the value S( f ) of an operatorHere we assume that X is a normed space of functions and G is also a normed space. The operator S describes the solution of a mathematical problem, for example the solution of a boundary value problem or an integral equation. Also, numerical integration (with G ϭ R) and the recovery of functions (with an imbedding S ϭ id: X Ǟ L p , where X ʚ L p ) can be stated in this way. In many cases the space X is infinite dimensional and therefore f ʦ X cannot directly be an input of a computation. We usually replace S with a discretization method given, for example, by a finite element method. Accordingly, numerical methods a...

Adaptive sensing performance lower bounds for sparse signal detection and support estimation

“…One of the key aspects of adaptive sensing is that the data collection process is sequential and adaptive. In different fields these sensing/experimenting paradigms are known by different names, such as sequential experimental design in statistics and economics (see Wald [35], Bessler [5], Fedorov [19], El-Gamal [18], Hall and Molchanov [21], Lai and Robbins [29], Blanchard and Geman [6]), active learning or adaptive sensing/sampling in computer science, engineering and machine learning (see Cohn, Ghahramani and Jordan [12], Freund et al [20], Novak [33], Korostelev and Kim [27], Dasgupta [13], Castro, Willett and Nowak [9], Dasgupta, Kalai and Monteleoni [15], Dasgupta [14], Hanneke [22], Koltchiinskii [28], Balcan, Beygelzimer and Langford [4], Castro and Nowak [10]). …”

Adaption allows efficient integration of functions with unknown singularities

“…There are theoretical results as well, however they hold for rather restrictive classes of functions or are probabilistic in nature, see, e.g. [9,14,20] and [15] for a survey.…”

Numerical Integration of Peak Functions

“…If we want to stress that I n is nonadaptive then we write I non n . Much is known about the power of adaption, see Traub et al (1988) and the recent survey Novak (1996).…”