Comparison of worst case errors in linear and neural network approximation

Kůrková, Véra; Sanguineti, Marcello

doi:10.1109/18.971754

Cited by 127 publications

(81 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…G-variation is a norm on the subspace {f ∈ X : f G < ∞} ⊆ X; for its properties see [15], [17], and [18]. In [16] and [18] it was shown that when G is an orthonormal basis of a separable Hilbert space, G-variation is equal to the l 1 -norm with respect to G, defined for f ∈ X as f 1,G = g∈G |f · g|.…”

Section: Rates Of Decrease Of Infima With Increasing Complexity Of Admentioning

confidence: 99%

“…Theorem 3.1 will be used in the next section to investigate generalized Tikhonov well-posedness of (M, e C ) for admissible sets M computable by variable-basis functions and, as a particular case, by neural networks. [16], [17]. Sets span n G model situations in which admissible functions are represented as linear combinations of any n-tuple of functions from G, with unconstrained coefficients in the linear combinations.…”

Section: Conditions On M and C Guaranteeing Tikhonov Well-posedness Imentioning

confidence: 99%

“…In the last decades, complex optimization problems of this kind have been approximately solved by searching suboptimal solutions over ad- Neural networks can be studied in a more general context of variable-basis functions, which also include other nonlinear families of functions such as free-node splines and trigonometric polynomials with free frequencies [17]. Families of variable-basis functions are formed by linear combinations of a fixed number of elements chosen from a given basis without a prespecified ordering [16], [17].When admissible functions depend on a large number of variables, implementation of some procedures of approximate optimization may be infeasible due to the "curse of dimensionality" [3]. For example, when optimization is performed over linear combinations of fixed-basis functions, the number of basis functions required to…”

mentioning

confidence: 99%

“…In the last decades, complex optimization problems of this kind have been approximately solved by searching suboptimal solutions over admissible sets of functions computable by neural networks [4], [21], [22], [25], [28], [29]. Neural networks can be studied in a more general context of variable-basis functions, which also include other nonlinear families of functions such as free-node splines and trigonometric polynomials with free frequencies [17]. Families of variable-basis functions are formed by linear combinations of a fixed number of elements chosen from a given basis without a prespecified ordering [16], [17].…”

mentioning

confidence: 99%

“…Neural networks can be studied in a more general context of variable-basis functions, which also include other nonlinear families of functions such as free-node splines and trigonometric polynomials with free frequencies [17]. Families of variable-basis functions are formed by linear combinations of a fixed number of elements chosen from a given basis without a prespecified ordering [16], [17].…”

mentioning

confidence: 99%

See 4 more Smart Citations

Minimization of Error Functionals over Variable-Basis Functions

Kainen¹,

Kurková²,

Sanguineti³

2004

SIAM J. Optim.

View full text Add to dashboard Cite

Abstract. Generalized Tikhonov well-posedness is investigated for the problem of minimization of error functionals over admissible sets formed by variable-basis functions, i.e., linear combinations of a fixed number of elements chosen from a given basis without a prespecified ordering. For variablebasis functions of increasing complexity, rates of decrease of infima of error functionals are estimated. Upper bounds are derived on such rates which do not exhibit the curse of dimensionality with respect to the number of variables of admissible functions. Consequences are considered for Boolean functions and decision trees. 1. Introduction. Functionals defined as distances from (target) sets are called error functionals. Minimization of such functionals occurs in optimization tasks arising in various areas such as decision processes, system identification, machine learning, and pattern recognition.In various applications, admissible solutions over which error functionals are minimized are functions depending on a large number of variables: for example, when routing strategies have to be devised for large-scale communication and transportation networks, when an optimal closed-loop control law has to be determined for a dynamical system with high-dimensional output measurement vector and a large number of decision stages, etc. In the last decades, complex optimization problems of this kind have been approximately solved by searching suboptimal solutions over ad- Neural networks can be studied in a more general context of variable-basis functions, which also include other nonlinear families of functions such as free-node splines and trigonometric polynomials with free frequencies [17]. Families of variable-basis functions are formed by linear combinations of a fixed number of elements chosen from a given basis without a prespecified ordering [16], [17].When admissible functions depend on a large number of variables, implementation of some procedures of approximate optimization may be infeasible due to the "curse of dimensionality" [3]. For example, when optimization is performed over linear combinations of fixed-basis functions, the number of basis functions required to

show abstract

Section: Rates Of Decrease Of Infima With Increasing Complexity Of Admentioning

confidence: 99%

Section: Conditions On M and C Guaranteeing Tikhonov Well-posedness Imentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

Minimization of Error Functionals over Variable-Basis Functions

Kainen¹,

Kurková²,

Sanguineti³

2004

SIAM J. Optim.

View full text Add to dashboard Cite

show abstract

Data‐driven forecasting of ship motions in waves using machine learning and dynamic mode decomposition

Diez,

Gaggero,

Serani

2024

Adaptive Control & Signal

View full text Add to dashboard Cite

SummaryData‐driven forecasting of ship motions in waves is investigated through feedforward and recurrent neural networks as well as dynamic mode decomposition. The goal is to predict future ship motion variables based on past data collected on the field, using equation‐free approaches. Numerical results in two case studies involving the course‐keeping of a naval destroyer in a high sea state using simulation data at model scale are presented. The proposed methods reveal successful in predicting ship motions both in short‐term and medium‐term perspectives with accuracy and reduced computational effort, thus enabling further advances in the identification, control, and optimization of ships operating in waves.

show abstract

Bounds on the complexity of neural‐network models and comparison with linear methods

Hlaváčková‐Schindler¹,

Sanguineti

2003

Adaptive Control & Signal

View full text Add to dashboard Cite

A class of non-linear models having the structure of combinations of simple, parametrized basis functions is investigated; this class includes widespread neural networks in which the basis functions correspond to the computational units of a type of networks. Bounds on the complexity of such models are derived in terms of the number of adjustable parameters necessary for a given modelling accuracy. These bounds guarantee a more advantageous tradeoff than linear methods between modelling accuracy and model complexity: the number of parameters may increase much more slowly, in some cases only polynomially, with the dimensionality of the input space in modelling tasks. Polynomial bounds on complexity allow one to cope with the so-called 'curse of dimensionality', which often makes linear methods either inaccurate or computationally unfeasible. The presented results let one gain a deeper theoretical insight into the effectiveness of neural-network architectures, noticed in complex modelling applications.

show abstract

Comparison of worst case errors in linear and neural network approximation

Cited by 127 publications

References 27 publications

Minimization of Error Functionals over Variable-Basis Functions

Minimization of Error Functionals over Variable-Basis Functions

Data‐driven forecasting of ship motions in waves using machine learning and dynamic mode decomposition

Bounds on the complexity of neural‐network models and comparison with linear methods

Contact Info

Product

Resources

About