On some neural network architectures that can represent viscosity solutions of certain high dimensional Hamilton–Jacobi partial differential equations

Darbon, Jérôme; Meng, Tingwei

doi:10.1016/j.jcp.2020.109907

Cited by 51 publications

(42 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…SciML includes a variety of techniques aiming at seamlessly combining observational data with available physical models. They can be divided into three main categories: (1) the ones that seek to emulate physical systems by utilizing large data amounts, such as the deep operator network (DeepONet) developed by Lu et al [6] (see also [7,8] for different versions of DeepONet and [9] for theoretical results, as well as [10] for a different approach termed Fourier neural operator); (2) the ones that encode physics into the NN architecture, such as the architectures developed by Darbon and Meng [11]; and (3) the ones that add physical soft constraints in the NN optimization process, such as the physics-informed neural network (PINN) developed by Raissi et al [12] (see also [13][14][15][16][17][18] for different versions of PINN and [19,20] for theoretical results). These techniques flourish, in principle, where the applicability of conventional solvers diminishes.…”

Section: Motivation and Scope Of The Papermentioning

confidence: 99%

Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons

Psaros¹,

Meng²,

Zou³

et al. 2022

Preprint

View full text Add to dashboard Cite

Neural networks (NNs) are currently changing the computational paradigm on how to combine data with mathematical laws in physics and engineering in a profound way, tackling challenging inverse and ill-posed problems not solvable with traditional methods. However, quantifying errors and uncertainties in NN-based inference is more complicated than in traditional methods. This is because in addition to aleatoric uncertainty associated with noisy data, there is also uncertainty due to limited data, but also due to NN hyperparameters, overparametrization, optimization and sampling errors as well as model misspecification. Although there are some recent works on uncertainty quantification (UQ) in NNs, there is no systematic investigation of suitable methods towards quantifying the total uncertainty effectively and efficiently even for function approximation, and there is even less work on solving partial differential equations and learning operator mappings between infinite-dimensional function spaces using NNs. In this work, we present a comprehensive framework that includes uncertainty modeling, new and existing solution methods, as well as evaluation metrics and post-hoc improvement approaches. To demonstrate the applicability and reliability of our framework, we present an extensive comparative study in which various methods are tested on prototype problems, including problems with mixed input-output data, and stochastic problems in high dimensions. In the Appendix, we include a comprehensive description of all the UQ methods employed, which we will make available as open-source library of all codes included in this framework.

show abstract

Section: Motivation and Scope Of The Papermentioning

confidence: 99%

Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons

Psaros¹,

Meng²,

Zou³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In [40], an approach for solving a certain kind of high‐dimensional first‐order Hamilton–Jacobi equations is proposed based the Hopf formula [77] whose computational expense behaves polynomially in the spatial dimension. In subsequent work, first‐order Hamilton–Jacobi equations in high dimension are considered in [38,39] based on classes of neural networks that exactly encode the viscosity solutions of these equations.…”

Section: Extensions and Related Workmentioning

confidence: 99%

Three ways to solve partial differential equations with neural networks — A review

2021

View full text Add to dashboard Cite

Neural networks are increasingly used to construct numerical solution methods for partial differential equations. In this expository review, we introduce and contrast three important recent approaches attractive in their simplicity and their suitability for high‐dimensional problems: physics‐informed neural networks, methods based on the Feynman–Kac formula and methods based on the solution of backward stochastic differential equations. The article is accompanied by a suite of expository software in the form of Jupyter notebooks in which each basic methodology is explained step by step, allowing for a quick assimilation and experimentation. An extensive bibliography summarizes the state of the art.

show abstract

“…In addition to the aforementioned data-driven machine learning techniques, physics-informed machine learning techniques, also known as physics-informed neural networks [156,157], have emerged as an alternative to illposed and inverse problems. Fundamental physical laws and domain knowledge are embedded by exploiting observational data [158,159], tailoring neural network architecture for physics constraints [160,161], and/or imposing physics constraints into the loss function [162,163]. The physics-informed neural network is expected to outperform existing machine learning methods in partially understood, uncertain, and high-dimensional problems.…”

Section: Metamodel-based Optimization Enhanced By Machine Learningmentioning

confidence: 99%

Metamodel-based multidisciplinary design optimization methods for aerospace system

Shi

et al. 2021

Astrodyn

View full text Add to dashboard Cite

The design of complex aerospace systems is a multidisciplinary design optimization (MDO) problem involving the interaction of multiple disciplines. However, because of the necessity of evaluating expensive black-box simulations, the enormous computational cost of solving MDO problems in aerospace systems has also become a problem in practice. To resolve this, metamodel-based design optimization techniques have been applied to MDO. With these methods, system models can be rapidly predicted using approximate metamodels to improve the optimization efficiency. This paper presents an overall survey of metamodel-based MDO for aerospace systems. From the perspective of aerospace system design, this paper introduces the fundamental methodology and technology of metamodel-based MDO, including aerospace system MDO problem formulation, metamodeling techniques, state-of-the-art metamodel-based multidisciplinary optimization strategies, and expensive black-box constraint-handling mechanisms. Moreover, various aerospace system examples are presented to illustrate the application of metamodel-based MDOs to practical engineering. The conclusions derived from this work are summarized in the final section of the paper. The survey results are expected to serve as guide and reference for designers involved in metamodel-based MDO in the field of aerospace engineering.

show abstract

On some neural network architectures that can represent viscosity solutions of certain high dimensional Hamilton–Jacobi partial differential equations

Cited by 51 publications

References 55 publications

Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons

Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons

Three ways to solve partial differential equations with neural networks — A review

Metamodel-based multidisciplinary design optimization methods for aerospace system

Contact Info

Product

Resources

About