Approximation by superpositions of a sigmoidal function

Abstract

In this paper we demonstrate that finite linear combinations of com positions of a fixed, univariate function and a set of affine functionals can uniformly approximate any continuous function of n real variables with support in the unit hypercube; only mild conditions are imposed on the univariate function. Our results settle an open question about representability in the class of single hidden layer neural networks. In particular, we show that arbitrary decision regions can be arbitrarily well approximated by continuous feedforward neural networks with only a single internal, hidden layer and any continuous sigmoidal nonlinearity. The paper discusses approximation properties of other possible types of nonlinearities that might be implemented by artificial neural networks.

Flavors of protein disorder

“…OLS and LR 32 are linear predictors, where OLS is suited for regression and is extremely fast to run, whereas LR is suited for binary classification and requires a slower (one order of magnitude) iterative parameter fitting procedure. NN are known as universal approximators, capable of discovering highly nonlinear relationships, 33 but they require abundant data, have relatively slow training, and are sensitive to the initial conditions and training procedures.…”

A mathematical model for file fragment diffusion and a neural predictor to manage priority queues over BitTorrent

“…The first property is a common characteristic of neural networks, since such solutions are universal approximators, as demonstrated by Cybenko (1989). For the latter property, we could use a mother wavelet as the transfer function; however, mother wavelets lack some elementary properties needed by a proper transfer function such as, e.g., the absence of local minima and a sufficiently graded and scaled response (Gupta et al, 2004).…”

Neural Networks: A General Framework for Non-Linear Function Approximation

“…Models of the form (2) represent a rich and flexible class of approximators. It is now well established that neural networks of the type (2) with linear output and sigmoid hidden layer transfer functions can approximate any continuous function uniformly on compacta, provided that sufficiently many hidden units are available (Cybenko 1989, Funahashi 1989, Hecht-Nielsen 1989, Hornik et al 1989). These results establish single hidden layer feedforward network models as a class of universal approximators.…”