On Best Approximation by Ridge Functions

Abstract

We consider best approximation of some function classes by the manifold M n consisting of sums of n arbitrary ridge functions. It is proved that the deviation of the Sobolev class W r, d 2 from the manifold M n in the space L 2 behaves asymptotically as n &rÂ(d&1) .

Lower bounds for approximation by MLP neural networks

“…In [13] (see also [18] for the case d"2) are determined upper and lower bounds on the degree of approximation from M L to some Sobolev-type spaces of functions with derivatives of all orders up to r in¸, de"ned on the unit ball in 1B. Without going into the details, as they are not relevant in what follows, they prove that for all functions in this set one can approximate in the¸norm from M L to within approximation error c n\P B\ , where c is some constant independent of n. It is also proven that for each n there exists a function in the set for which one cannot approximate from M L with approximation error less than c n\P B\ for some other constant c independent of n. In [14] it is shown that the set of functions for which this lower bound holds is of large measure.…”

“…Without going into the details, as they are not relevant in what follows, they prove that for all functions in this set one can approximate in the¸norm from M L to within approximation error c n\P B\ , where c is some constant independent of n. It is also proven that for each n there exists a function in the set for which one cannot approximate from M L with approximation error less than c n\P B\ for some other constant c independent of n. In [14] it is shown that the set of functions for which this lower bound holds is of large measure. See [13,14] for details. The point we wish to make here is twofold.…”

Geometric properties of the ridge function manifold

“…This is closely related to the estimation of the number of connected components of polynomial manifolds as is done for instance in [2,7,13,14]. Let p be a real polynomial from the space P ν s , i.e., a polynomial on ν variables of degree at most s. The set of points x ∈ R ν on which a polynomial p vanishes will be denoted by Z ( p).…”

“…Let n > s d−1 . Denote by Q d s the subspace in P d s consisting of all homogeneous polynomials of degree s. We know (see [7]) that…”

Fields of nonlinear regression models for inversion of satellite data

“…Models such as perceptrons, falling in the class of so-called ridge constructions, achieve this statistical modeling goal with several well-known interesting properties. Let us just mention the density or universal approximation property [Cybenko, 1989;Lin and Pinkus, 1993], and the results related to the approximation rate, including the dimension-independent upper bound [Barron, 1993;Burger and Neubauer, 2001;Makovoz, 1998], and the asymptotic expression obtained by Maiorov [1999].…”