Some comparisons of complexity in dictionary-based and linear computational models

Gnecco, Giorgio; Krková, Vra; Sanguineti, Marcello

doi:10.1016/j.neunet.2010.10.002

Cited by 22 publications

(14 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…the error is proportional with n b d / , where n is the number of neurons, d is the dimensionality of the data and 1  b is a constant depending on the class of the activation functions [8,16]. We note that in particular cases, the approximation error bounds are smaller and do not grow exponentially with the dimensionality of the data [17], however these special cases do not apply very often.…”

Section: Related Workmentioning

confidence: 94%

High-Dimensional Function Approximation With Neural Networks for Large Volumes of Data

András

2018

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

Section: Related Workmentioning

confidence: 94%

High-Dimensional Function Approximation With Neural Networks for Large Volumes of Data

András

2018

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

“…The approximation theory results typically imply a relatively high minimal number of neurons for the worst case scenario approximation [31][32][33][34][35]. Some of these results show that the number of required neurons in many cases grows exponentially or polinomially with the dimensionality of the data [18,32,[35][36][37]. However if the parameters of the basis functions represented by the neurons are allowed to vary, for certain classes of approximated functions and appropriate basis functions, the required number of neurons may not be required to grow so quickly or at all with the dimensionality of the data [36][37][38].…”

Section: B Supervised Learning Of Function Approximation With Singlementioning

confidence: 99%

“…In general, key problems of nonlinear function approximation with neural networks remain that the reliable approximation of functions defined on high dimensional inputs often requires very large data volumes that grow exponentially with the dimensionality of the data [18,36,37,43], and finding simple models of the data with good generalization ability, in the form of neurons networks is very difficult. Often the volume of the available data is very small if the dimensionality of the data is taken into account, i.e.…”

Section: B Supervised Learning Of Function Approximation With Singlementioning

confidence: 99%

Function Approximation Using Combined Unsupervised and Supervised Learning

András

2014

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

Function approximation is one of the core tasks that are solved using neural networks in the context of many engineering problems. However, good approximation results need good sampling of the data space, which usually requires exponentially increasing volume of data as the dimensionality of the data increases. At the same time, often the high-dimensional data is arranged around a much lower dimensional manifold. Here we propose the breaking of the function approximation task for high-dimensional data into two steps: (1) the mapping of the high-dimensional data onto a lower dimensional space corresponding to the manifold on which the data resides and (2) the approximation of the function using the mapped lower dimensional data. We use over-complete self-organizing maps (SOMs) for the mapping through unsupervised learning, and single hidden layer neural networks for the function approximation through supervised learning. We also extend the two-step procedure by considering support vector machines and Bayesian SOMs for the determination of the best parameters for the nonlinear neurons in the hidden layer of the neural networks used for the function approximation. We compare the approximation performance of the proposed neural networks using a set of functions and show that indeed the neural networks using combined unsupervised and supervised learning outperform in most cases the neural networks that learn the function approximation using the original high-dimensional data.

show abstract

“…Thus they are sometimes called variable-basis approximation schemas in contrast to traditional linear approximators which are called fixed-basis approximation schemas. In some cases, especially in the case of functions of large numbers of variables, neural networks achieve better approximation rates than linear models with much smaller model complexity [9,10]. Motivated by the experimental exploration [8,7] of surrogate models of solutions of Fredholm equations by neural networks, Gnecco et al [11] initiated a theoretical analysis of this modelling.…”

Section: Introductionmentioning

confidence: 99%