Conservative physics-informed neural networks on discrete domains for conservation laws: Applications to forward and inverse problems

Jagtap, Ameya D.; Kharazmi, Ehsan; Karniadakis, George Em

doi:10.1016/j.cma.2020.113028

Cited by 657 publications

(279 citation statements)

References 16 publications

Supporting

Mentioning

278

Contrasting

Order By: Relevance

“…The main feature of the PINN is that it can easily incorporate all the given information like governing equation, experimental data, initial/boundary conditions, etc., into the loss function thereby recast the original problem into an optimization problem. One of the main limitation of PINN algorithm is its high computational cost for high-dimensional optimization problem, which is addressed in [13] by employing the domain decomposition approach. The PINN algorithm aims to learn a surrogate u=uboldΘfalse^ for predicting the solution u of the governing PDE.…”

Section: Methodsmentioning

confidence: 99%

Locally adaptive activation functions with slope recovery for deep and physics-informed neural networks

2020

Self Cite

View full text Add to dashboard Cite

We propose two approaches of locally adaptive activation functions namely, layer-wise and neuron-wise locally adaptive activation functions, which improve the performance of deep and physics-informed neural networks. The local adaptation of activation function is achieved by introducing a scalable parameter in each layer (layer-wise) and for every neuron (neuron-wise) separately, and then optimizing it using a variant of stochastic gradient descent algorithm. In order to further increase the training speed, an activation slope-based slope recovery term is added in the loss function, which further accelerates convergence, thereby reducing the training cost. On the theoretical side, we prove that in the proposed method, the gradient descent algorithms are not attracted to sub-optimal critical points or local minima under practical conditions on the initialization and learning rate, and that the gradient dynamics of the proposed method is not achievable by base methods with any (adaptive) learning rates. We further show that the adaptive activation methods accelerate the convergence by implicitly multiplying conditioning matrices to the gradient of the base method without any explicit computation of the conditioning matrix and the matrix–vector product. The different adaptive activation functions are shown to induce different implicit conditioning matrices. Furthermore, the proposed methods with the slope recovery are shown to accelerate the training process.

show abstract

Section: Methodsmentioning

confidence: 99%

Locally adaptive activation functions with slope recovery for deep and physics-informed neural networks

2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…In addition, PINNs have been applied successfully in a wide range of applications, including fluid dynamics [113,115,117,160,177], continuum mechanics and elastodynamics [66,132,162], inverse problems [91,121], fractional advection–diffusion equations [135], stochastic advection–diffusion–reaction equations [34], stochastic differential equations [179] and power systems [127]. Finally, we mention that Gaussian processes as an alternative to neural networks for approximating complex multivariate functions have also been studied extensively for solving PDEs and inverse problems [136,155,158,164].…”

Section: Physics‐informed Neural Networkmentioning

confidence: 99%

Three ways to solve partial differential equations with neural networks — A review

2021

View full text Add to dashboard Cite

Neural networks are increasingly used to construct numerical solution methods for partial differential equations. In this expository review, we introduce and contrast three important recent approaches attractive in their simplicity and their suitability for high‐dimensional problems: physics‐informed neural networks, methods based on the Feynman–Kac formula and methods based on the solution of backward stochastic differential equations. The article is accompanied by a suite of expository software in the form of Jupyter notebooks in which each basic methodology is explained step by step, allowing for a quick assimilation and experimentation. An extensive bibliography summarizes the state of the art.

show abstract

“…The basic interface conditions for XPINN include the residual continuity condition in strong form as well as enforcing the average solution given by different Sub-Net's along the common interface. As discussed in the cPINN framework [46], for stability it is not necessary to enforce the average solution along the common interface, but the compu-tational experiments reveal that it will drastically speed-up the convergence rate. Fig.…”

Section: Extended Physics-informed Neural Networkmentioning

confidence: 99%

“…Therefore, it is crucial to accelerate the convergence of such models without sacrificing the performance. This issue was first addressed in the conservative PINN (cPINN) method for conservation laws [46] by employing the domain decomposition approach in PINN framework. Domain decomposition has been a fundamental development in standard numerical methods, e.g.…”

Section: Introductionmentioning

confidence: 99%

Extended Physics-Informed Neural Networks (XPINNs): A Generalized Space-Time Domain Decomposition Based Deep Learning Framework for Nonlinear Partial Differential Equations

Karniadakis¹

2020

CICP

426

View full text Add to dashboard Cite

Conservative physics-informed neural networks on discrete domains for conservation laws: Applications to forward and inverse problems

Cited by 657 publications

References 16 publications

Locally adaptive activation functions with slope recovery for deep and physics-informed neural networks

Locally adaptive activation functions with slope recovery for deep and physics-informed neural networks

Three ways to solve partial differential equations with neural networks — A review

Extended Physics-Informed Neural Networks (XPINNs): A Generalized Space-Time Domain Decomposition Based Deep Learning Framework for Nonlinear Partial Differential Equations

Contact Info

Product

Resources

About