Multilayer feedforward networks with a nonpolynomial activation function can approximate any function

Leshno, Moshe; Lin, Vladimir; Pinkus, Allan; Schocken, Shimon

doi:10.1016/s0893-6080(05)80131-5

Cited by 1,806 publications

(1,018 citation statements)

References 13 publications

Supporting

Mentioning

946

Contrasting

Unclassified

Order By: Relevance

“…Such representation closely corresponds to the Kolmogorov function superposition theorem [55]. Basing on this relation it was shown [57,58] that the MLP can approximate any continuous function of its inputs, to the extent that depends on the number of the hidden neurons. However, in the practical problem we are faced, the desired function is not known and only the limited number of experimental points is available instead.…”

Section: Multi-layer Perceptronmentioning

confidence: 68%

Neural network parameterizations of electromagnetic nucleon form-factors

Graczyk

Płoński

Sulej

2010

J. High Energ. Phys.

View full text Add to dashboard Cite

Abstract:The electromagnetic nucleon form-factors data are studied with artificial feed forward neural networks. As a result the unbiased model-independent form-factor parametrizations are evaluated together with uncertainties. The Bayesian approach for the neural networks is adapted for χ 2 error-like function and applied to the data analysis. The sequence of the feed forward neural networks with one hidden layer of units is considered. The given neural network represents a particular form-factor parametrization. The so-called evidence (the measure of how much the data favor given statistical model) is computed with the Bayesian framework and it is used to determine the best form factor parametrization.

show abstract

Section: Multi-layer Perceptronmentioning

confidence: 68%

Neural network parameterizations of electromagnetic nucleon form-factors

Graczyk

Płoński

Sulej

2010

J. High Energ. Phys.

View full text Add to dashboard Cite

show abstract

“…The feed-forward neural network, by far the most widely used in practice, is essentially a highly non-linear function representation formed by repeatedly combining a fixed non-linear transfer function. It has been shown in [113] that any real-valued continuous function over R d can be arbitrarily well approximated by a feed-forward neural network with a single hidden layer, as long as the transfer function is not a polynomial. Given the potential power of neural networks in representing arbitrary continuous functions, it would seem that they could easily lead to overfitting and not work effectively in the context of Boosting.…”

Section: On the Choice Of Weak Learners For Boostingmentioning

confidence: 99%

An Introduction to Boosting and Leveraging

Meir

Rätsch

2003

Advanced Lectures on Machine Learning

281

186

View full text Add to dashboard Cite

Abstract. We provide an introduction to theoretical and practical aspects of Boosting and Ensemble learning, providing a useful reference for researchers in the field of Boosting as well as for those seeking to enter this fascinating area of research. We begin with a short background concerning the necessary learning theoretical foundations of weak learners and their linear combinations. We then point out the useful connection between Boosting and the Theory of Optimization, which facilitates the understanding of Boosting and later on enables us to move on to new Boosting algorithms, applicable to a broad spectrum of problems. In order to increase the relevance of the paper to practitioners, we have added remarks, pseudo code, "tricks of the trade", and algorithmic considerations where appropriate. Finally, we illustrate the usefulness of Boosting algorithms by giving an overview of some existing applications. The main ideas are illustrated on the problem of binary classification, although several extensions are discussed.

show abstract

“…The most common activation functions ' i (! ; b; x) in the hidden units are sigmoidal for Multi-layer Perceptrons (MLP) and radially symmetric for Radial Basis Function Networks (RBFN), although many other functions may be used [19,24]. Output activation functions ' 0 (u) are usually sigmoidal or linear.…”

Section: Feed-forward Neural Network (Fnn)mentioning

confidence: 99%