A Priori Analysis of Stable Neural Network Solutions to Numerical PDEs

Hong, Qingguo; Siegel, Jonathan W.; Xu, Jinchao

doi:10.48550/arxiv.2104.02903

Cited by 12 publications

(21 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Greedy algorithms for expanding a function f ∈ H as a linear combination of the dictionary elements are fundamental in approximation theory [11,41,40] and signal processing [29,30]. Greedy methods have also been proposed for optimizing shallow neural networks [23,9] and recently such methods have been proposed for training shallow neural networks to solve PDEs [17], which is the approach we take here.…”

Section: Problem Setupmentioning

confidence: 99%

“…This algorithm was first introduced and analyzed by Jones [18] for function approximation (i.e. J (u) = u − f 2 H ), and has been extended to the optimization of general convex objectives as well [46,17]. The convergence theorem we will use in our analysis is the following.…”

Section: Relaxed Greedy Algorithmmentioning

confidence: 99%

“…Theorem 1 (Theorem 1 in [17]). Suppose that the dictionary D is symmetric and satisfies sup d∈D d H ≤ C < ∞.…”

Section: Relaxed Greedy Algorithmmentioning

confidence: 99%

“…On the other hand, networks which are small enough or satisfy an appropriate bound on their coefficients to guarantee generalization cannot be provably optimized using gradient descent or its variants. Recently, this gap has been closed for shallow neural networks in [17]. The key idea is to use a greedy algorithm to train shallow neural networks instead of gradient descent.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

An efficient greedy training algorithm for neural networks and applications in PDEs

Siegel¹,

Hong²,

Jin³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Recently, neural networks have been widely applied for solving partial differential equations. However, the resulting optimization problem brings many challenges for current training algorithms. This manifests itself in the fact that the convergence order that has been proven theoretically cannot be obtained numerically. In this paper, we develop a novel greedy training algorithm for solving PDEs which builds the neural network architecture adaptively. It is the first training algorithm that observes the convergence order of neural networks numerically. This innovative algorithm is tested on several benchmark examples in both 1D and 2D to confirm its efficiency and robustness.

show abstract

Section: Problem Setupmentioning

confidence: 99%

Section: Relaxed Greedy Algorithmmentioning

confidence: 99%

“…Theorem 1 (Theorem 1 in [17]). Suppose that the dictionary D is symmetric and satisfies sup d∈D d H ≤ C < ∞.…”

Section: Relaxed Greedy Algorithmmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An efficient greedy training algorithm for neural networks and applications in PDEs

Siegel¹,

Hong²,

Jin³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…For PINNs, the convergence analysis is provided in [51,62,63], and a PINN with ReLu 3 network is analyzed and the convergence rate was given in C 2 norm ( [35]). The error analysis of DRM was established in [47,68,32,46] via assuming that the exact solution is contained in the spectral Barron space which has the property of being approximated by a two-layer neural network. The convergence rate of DRM with smooth activation functions like logistic or hyperbolic tangent was derived in H 1 norm for elliptic equations ( [20,36]).…”

Section: Introductionmentioning

confidence: 99%

Deep Petrov-Galerkin Method for Solving Partial Differential Equations

Shang¹,

Wang²,

Sun³

2022

Preprint

View full text Add to dashboard Cite

Deep neural networks are powerful tools for approximating functions, and they are applied to successfully solve various problems in many fields. In this paper, we propose a neural network-based numerical method to solve partial differential equations. In this new framework, the method is designed on weak formulations, and the unknown functions are approximated by deep neural networks and test functions can be chosen by different approaches, for instance, basis functions of finite element methods, neural networks, and so on. Because the spaces of trial function and test function are different, we name this new approach by Deep Petrov-Galerkin Method (DPGM). The resulted linear system is not necessarily to be symmetric and square, so the discretized problem is solved by a least-square method. Take the Poisson problem as an example, mixed DPGMs based on several mixed formulations are proposed and studied as well.In addition, we apply the DPGM to solve two classical time-dependent problems based on the space-time approach, that is, the unknown function is approximated by a neural network, in which temporal variable and spatial variables are treated equally, and the initial conditions are regarded as boundary conditions for the space-time domain. Finally, several numerical examples are presented to show the performance of the DPGMs, and we observe that this new method outperforms traditional numerical methods in several aspects: compared to the finite element method and finite difference method, DPGM is much more accurate with respect to degrees of freedom; this method is mesh-free, and can be implemented easily; mixed DPGM has good flexibility to handle different boundary conditions; DPGM can solve the time-dependent problems by the space-time approach naturally and efficiently. The proposed deep Petrov-Galerkin method shows strong potential in the field of numerical methods for partial differential equations.

show abstract

Imaging conductivity from current density magnitude using neural networks*

Jin

2022

Inverse Problems

View full text Add to dashboard Cite

Conductivity imaging represents one of the most important tasks in medical imaging. In this work we develop a neural network based reconstruction technique for imaging the conductivity from the magnitude of the internal current density. It is achieved by formulating the problem as a relaxed weighted least-gradient problem, and then approximating its minimizer by standard fully connected feedforward neural networks. We derive bounds on two components of the generalization error, i.e., approximation error and statistical error, explicitly in terms of properties of the neural networks (e.g., depth, total number of parameters, and the bound of the network parameters). We illustrate the performance and distinct features of the approach on several numerical experiments. Numerically, it is observed that the approach enjoys remarkable robustness with respect to the presence of data noise.

show abstract

A Priori Analysis of Stable Neural Network Solutions to Numerical PDEs

Cited by 12 publications

References 33 publications

An efficient greedy training algorithm for neural networks and applications in PDEs

An efficient greedy training algorithm for neural networks and applications in PDEs

Deep Petrov-Galerkin Method for Solving Partial Differential Equations

Imaging conductivity from current density magnitude using neural networks*

Contact Info

Product

Resources

About