Approximation by ridge functions and neural networks with a bounded number of neurons

A note on the equioscillation theorem for best ridge function approximation

“…Remark. Theorem 2.1 was proven by Ismailov [12] and in a more general form by Pinkus [33] under additional assumption that Q is convex. Convexity assumption was made to guarantee continuity of the following functions where F is an arbitrary continuous function on Q.…”

“…In [21], he obtained an equioscillation theorem for a best approximating sum ϕ(x) + ψ(y). In our papers [12,16], Chebyshev type theorems were proven for ridge functions under additional assumption that Q is convex. For a more recent and detailed discussion of an equioscillation theorem in ridge function approximation see Pinkus [33].…”

Approximation capability of two hidden layer feedforward neural networks with fixed weights

“…In recent years, the theory of neural networks has been developed further in this direction. For example, from the point of view of practical applications, SLFNs with a restricted set of weights have gained a special interest (see, e.g., [9,18,20,21,24,30]). It was proved that SLFNs with some restricted set of weights still possess the universal approximation property.…”

“…Ito [22,23] investigated this property of networks using monotone sigmoidal functions, with only weights located on the unit sphere. In [18,20,21], the second coauthor considered SLFNs with weights varying on a restricted set of directions, and gave several necessary and sufficient conditions for good approximation by such networks. For a set of weights consisting of two directions, he showed that there is a geometrically explicit solution to the problem.…”

On the error of approximation by ridge functions with two fixed directions

“…In other words, a ridge function is a multivariate function constant on the parallel hyperplanes a · x = c, c ∈ R. These functions and their linear combinations arise naturally in problems of computerized tomography (see, e.g., [26,31]), statistics (see, e.g., [5,9,10,15]), partial differential equations [24] (where they are called plane waves), neural networks (see, e.g., [6,16,33,35] and references therein), and approximation theory (see, e.g., [6,7,13,19,21,25,27,32,33,34,37]). …”