Exact imposition of boundary conditions with distance functions in physics-informed deep neural networks

Sukumar, N.; Srivastava, Ankit

doi:10.48550/arxiv.2104.08426

Cited by 4 publications

(3 citation statements)

References 85 publications

(237 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For time-dependent problems, we simply concatenate the time coordinates t with the constructed Fourier features embedding, i.e., u θ ([t, v(x)]), or u θ ([t, v(x, y)]). Although in this work we will only consider periodic problems, other types of boundary conditions, including Dirichlet, Neumann, Robin, etc., can also be enforced in a "hard" manner, see [45,46] for more details.…”

Section: Practical Considerationsmentioning

confidence: 99%

Respecting causality is all you need for training physics-informed neural networks

Wang¹,

Sankaran²,

Perdikaris³

2022

Preprint

View full text Add to dashboard Cite

While the popularity of physics-informed neural networks (PINNs) is steadily rising, to this date PINNs have not been successful in simulating dynamical systems whose solution exhibits multi-scale, chaotic or turbulent behavior. In this work we attribute this shortcoming to the inability of existing PINNs formulations to respect the spatio-temporal causal structure that is inherent to the evolution of physical systems. We argue that this is a fundamental limitation and a key source of error that can ultimately steer PINN models to converge towards erroneous solutions. We address this pathology by proposing a simple re-formulation of PINNs loss functions that can explicitly account for physical causality during model training. We demonstrate that this simple modification alone is enough to introduce significant accuracy improvements, as well as a practical quantitative mechanism for assessing the convergence of a PINNs model. We provide state-of-the-art numerical results across a series of benchmarks for which existing PINNs formulations fail, including the chaotic Lorenz system, the Kuramoto-Sivashinsky equation in the chaotic regime, and the Navier-Stokes equations in the turbulent regime. To the best of our knowledge, this is the first time that PINNs have been successful in simulating such systems, introducing new opportunities for their applicability to problems of industrial complexity.

show abstract

Section: Practical Considerationsmentioning

confidence: 99%

Respecting causality is all you need for training physics-informed neural networks

Wang¹,

Sankaran²,

Perdikaris³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Domain decomposition can be used for problems in a large domain [17,18,19]. Neural network architectures can also be modified to satisfy automatically and exactly the required Dirichlet boundary conditions [20,21,22], Neumann boundary conditions [23,24], Robin boundary conditions [25], periodic boundary conditions [26,8], and interface conditions [25]. In addition, if some features of the PDE solutions are known a-priori, it is also possible to encode them in network architectures, for example, multi-scale and high-frequency features [27,28,29,10,30].…”

Section: Introductionmentioning

confidence: 99%

Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems

Yu,

Lu,

Meng

et al. 2021

Preprint

View full text Add to dashboard Cite

Deep learning has been shown to be an effective tool in solving partial differential equations (PDEs) through physics-informed neural networks (PINNs). PINNs embed the PDE residual into the loss function of the neural network, and have been successfully employed to solve diverse forward and inverse PDE problems. However, one disadvantage of the first generation of PINNs is that they usually have limited accuracy even with many training points. Here, we propose a new method, gradient-enhanced physics-informed neural networks (gPINNs), for improving the accuracy and training efficiency of PINNs. gPINNs leverage gradient information of the PDE residual and embed the gradient into the loss function. We tested gPINNs extensively and demonstrated the effectiveness of gPINNs in both forward and inverse PDE problems. Our numerical results show that gPINN performs better than PINN with fewer training points. Furthermore, we combined gPINN with the method of residual-based adaptive refinement (RAR), a method for improving the distribution of training points adaptively during training, to further improve the performance of gPINN, especially in PDEs with solutions that have steep gradients.

show abstract

“…Furthermore, there also exhibit multiple ways to express the physical constraints or governing equations for both forward and inverse problems including the collocation-based loss function [Dissanayake and Phan-Thien, 1994, Lagaris et al, 1998, Sirignano and Spiliopoulos, 2018, Raissi, 2018, Raissi et al, 2019, Xu and Darve, 2019, which evaluate the solution at selected collocation points, and the energy-based (Ritz-Galerkin) method that requires numerical integrations but also reduces the order of the derivatives in the governing equations, [Weinan andYu, 2018, Samaniego et al, 2020], and the related variational approach Kharazmi et al [2021] that parametrizes trial and test spaces by neural network and polynomials, respectively. This vast number of choices is further complicated by the large number of tunable hyperparameters, such as the configurations of the neural network [Fuchs et al, 2021, Psaros et al, 2021, the types of activation functions [Psaros et al, 2021] and the neuron weight initialization [Glorot and Bengio, 2010, He et al, 2015, Goodfellow et al, 2016, Cyr et al, 2020, and different techniques to impose boundary conditions [Sukumar and Srivastava, 2021] , while providing significant flexibility, also could make it confusing for researchers unfamiliar with neural network to determine the optimal way to train the PINN properly and efficiently. Finally, the recent work on meta-learning and analysis on the enforcement of boundary conditions also reveals that the multiple physical constraints employed to measure and minimize the error of the approximated solution may lead to conflicting situations where actions that reduce one set of constraints may also increase the error of another set of constraints and lead to a comprised local minimizer that is not desirable [Psaros et al, 2021, Yu et al, 2020, Rohrhofer et al, 2021.…”

Section: Introductionmentioning

confidence: 99%

Training multi-objective/multi-task collocation physics-informed neural network with student/teachers transfer learnings

Bahmani,

Sun

2021

Preprint

View full text Add to dashboard Cite

This paper presents a PINN training framework that employs (1) pre-training steps that accelerates and improve the robustness of the training of physics-informed neural network with auxiliary data stored in point clouds, (2) a net-to-net knowledge transfer algorithm that improves the weight initialization of the neural network and (3) a multi-objective optimization algorithm that may improve the performance of a physical-informed neural network with competing constraints. We consider the training and transfer and multi-task learning of physics-informed neural network (PINN) as multi-objective problems where the physics constraints such as the governing equation, boundary conditions, thermodynamic inequality, symmetry, and invariant properties, as well as point cloud used for pre-training can sometimes lead to conflicts and necessitating the seek of the Pareto optimal solution. In these situations, weighted norms commonly used to handle multiple constraints may lead to poor performance, while other multi-objective algorithms may scale poorly with increasing dimensionality. To overcome this technical barrier, we adopt the concept of vectorized objective function and modify a gradient descent approach to handle the issue of conflicting gradients. Numerical experiments are compared the benchmark boundary value problems solved via PINN. The performance of the proposed paradigm is compared against the classical equal-weighted norm approach. Our numerical experiments indicate that the brittleness and lack of robustness demonstrated in some PINN implementations can be overcome with the proposed strategy.

show abstract

Exact imposition of boundary conditions with distance functions in physics-informed deep neural networks

Cited by 4 publications

References 85 publications

Respecting causality is all you need for training physics-informed neural networks

Respecting causality is all you need for training physics-informed neural networks

Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems

Training multi-objective/multi-task collocation physics-informed neural network with student/teachers transfer learnings

Contact Info

Product

Resources

About