Lower bounds for artificial neural network approximations: A proof that shallow neural networks fail to overcome the curse of dimensionality

Shokhrukh, Ibragimov,; Jentzen, Arnulf; Koppensteiner, Sarah

doi:10.1016/j.jco.2023.101746

Cited by 8 publications

(6 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In particular, in [75] it is proven that NNs have the expressiveness to overcome the CoD for semilinear heat PDEs. On the other hand, there are also some articles, [51,54,136,165] that derive some lower bounds on the complexity of NNs with rectified linear unit (ReLU) activation function to achieve a certain accuracy, showing that there are natural classes of functions where deep NNs with ReLU activation cannot escape the CoD.…”

Section: Approximation Results For Solution Learning In Pdesmentioning

confidence: 99%

Deep learning methods for partial differential equations and related parameter identification problems

Nganyu Tanyu,

Ning,

Freudenberg

et al. 2023

Inverse Problems

View full text Add to dashboard Cite

Recent years have witnessed a growth in mathematics for deep learning—which seeks a deeper understanding of the concepts of deep learning with mathematics and explores how to make it more robust—and deep learning for mathematics, where deep learning algorithms are used to solve problems in mathematics. The latter has popularised the field of scientific machine learning where deep learning is applied to problems in scientific computing. Specifically, more and more neural network architectures have been developed to solve specific classes of partial differential equations (PDEs). Such methods exploit properties that are inherent to PDEs and thus solve the PDEs better than standard feed-forward neural networks, recurrent neural networks, or convolutional neural networks. This has had a great impact in the area of mathematical modeling where parametric PDEs are widely used to model most natural and physical processes arising in science and engineering. In this work, we review such methods as well as their extensions for parametric studies and for solving the related inverse problems. We equally proceed to show their relevance in some industrial applications.

show abstract

Section: Approximation Results For Solution Learning In Pdesmentioning

confidence: 99%

Deep learning methods for partial differential equations and related parameter identification problems

Nganyu Tanyu,

Ning,

Freudenberg

et al. 2023

Inverse Problems

View full text Add to dashboard Cite

show abstract

“…(2020); Grohs et al. (2022) have shown that feedforward neural networks can efficiently approximate most SDE's associated PDE. Likewise, regular path‐functionals of a jump‐diffusion process with Lipschitz coefficients can efficiently be approximated by neural SDEs Gonon and Schwab (2021).…”

Section: Dynamic Case – Universal Approximation Of Causal Mapsmentioning

confidence: 99%

Designing universal causal deep learning models: The geometric (Hyper)transformer

Acciaio

Kratsios

Pammer

2023

Mathematical Finance

View full text Add to dashboard Cite

Several problems in stochastic analysis are defined through their geometry, and preserving that geometric structure is essential to generating meaningful predictions. Nevertheless, how to design principled deep learning (DL) models capable of encoding these geometric structures remains largely unknown. We address this open problem by introducing a universal causal geometric DL framework in which the user specifies a suitable pair of metric spaces and and our framework returns a DL model capable of causally approximating any “regular” map sending time series in to time series in while respecting their forward flow of information throughout time. Suitable geometries on include various (adapted) Wasserstein spaces arising in optimal stopping problems, a variety of statistical manifolds describing the conditional distribution of continuous‐time finite state Markov chains, and all Fréchet spaces admitting a Schauder basis, for example, as in classical finance. Suitable spaces are compact subsets of any Euclidean space. Our results all quantitatively express the number of parameters needed for our DL model to achieve a given approximation error as a function of the target map's regularity and the geometric structure both of and of . Even when omitting any temporal structure, our universal approximation theorems are the first guarantees that Hölder functions, defined between such and can be approximated by DL models.

show abstract

“…PDE from computational finance, the Hamilton-Jacobi-Bellmann (HJB) PDEs arising from stochastic optimal control problems, or the many-electron Schrödinger equation from computational chemistry. For BS PDEs and certain nonlinear HJB PDEs we were recently able to prove that neural networks are capable of representing their solutions without incurring the curse of dimensionality [9,10], and that such solutions can be numerically found by solving an empirical risk minimization (ERM) problem of a size scaling only polynomially in the problem dimension [3]. While the analysis of the computational complexity of the ERM problem remains wide open, there are some empirical results suggesting that its scaling does not suffer from the curse of dimensionality either, see [2] and Figure 3.…”

Section: Scientific Machine Learningmentioning

confidence: 99%

RICAM, the Johann Radon Institute for Computational and Applied Mathematics

Kritzer

Kunisch

Ramlau

et al. 2021

Eur. Math. Soc. Mag.

Self Cite

View full text Add to dashboard Cite

RICAM) of the Austrian Academy of Sciences focuses on basic research in applied mathematics. The institute is based inAustria's third-largest city and industrial hub, Linz. Researchers from all around the globe collaborate on common core areas in mathematical modeling, simulation, inverse problems, and optimization. RICAM stands for excellence in research, as can be seen from a high level of publications and the popularity of the Institute's Special Semesters within the academic community. The work groups at RICAM provide a broad field of expertise over a whole range of different subjects, and together they create an exciting atmosphere to carry out research in applied mathematics.

show abstract

Lower bounds for artificial neural network approximations: A proof that shallow neural networks fail to overcome the curse of dimensionality

Cited by 8 publications

References 42 publications

Deep learning methods for partial differential equations and related parameter identification problems

Deep learning methods for partial differential equations and related parameter identification problems

Designing universal causal deep learning models: The geometric (Hyper)transformer

RICAM, the Johann Radon Institute for Computational and Applied Mathematics

Contact Info

Product

Resources

About