Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next

Abstract

Physics-Informed Neural Networks (PINN) are neural networks (NNs) that encode model equations, like Partial Differential Equations (PDE), as a component of the neural network itself. PINNs are nowadays used to solve PDEs, fractional equations, integral-differential equations, and stochastic PDEs. This novel methodology has arisen as a multi-task learning framework in which a NN must fit observed data while reducing a PDE residual. This article provides a comprehensive review of the literature on PINNs: while the primary goal of the study was to characterize these networks and their related advantages and disadvantages. The review also attempts to incorporate publications on a broader range of collocation-based physics informed neural networks, which stars form the vanilla PINN, as well as many other variants, such as physics-constrained neural networks (PCNN), variational hp-VPINN, and conservative PINN (CPINN). The study indicates that most research has focused on customizing the PINN through different activation functions, gradient optimization techniques, neural network structures, and loss function structures. Despite the wide range of applications for which PINNs have been used, by demonstrating their ability to be more feasible in some contexts than classical numerical techniques like Finite Element Method (FEM), advancements are still possible, most notably theoretical issues that remain unresolved.

A Modified PINN Approach for Identifiable Compartmental Models in Epidemiology with Applications to COVID-19

“…For example, recurrent neural networks like the Long Short Term Memory (LSTM) network have been used to improve pattern recognition in data that has historical dependencies [19]. The PINN approach involves the changing of the loss function for the network, rather than any particular change in the network architecture itself, and in fact is compatible with a wide variety of possible architectures [10,23]. The concept of the PINN is to incorporate some physical law that must be obeyed by the initial system, and to introduce an extra term into the loss function which becomes smaller the closer the network output adheres to the law [32].…”

“…Before we precisely define our modified approach, we first review the standard PINN implementation. Generally, although many forms of PINN systems have been used, the loss function is basically of the same form [10,23]. To give a precise definition for the ODE case, consider an n-dimensional ODE system defined by Ẋ = f (X, p), (4.3) where f := f (X, p) is some function of all our variables X, and the given system parameters.…”

“…We propose the usage of a sophisticated neural network approach to estimate the unknown cases and parameters, i.e., a methodology that has proved extremely successful in a variety of applications within scientific computing [23,10]. There have been promising recent results along this vein through the usage of neural networks for forecasting the development of COVID-19 in Germany [15] and the spread of influenza in the U.S. [3].…”

“…The principle of PINNs was developed for the estimation of unknown quantities in a system adhering to a physical law, generally a nonlinear PDE (or lattice dynamical equation [44]) that the system is assumed to obey [32]. The effectiveness of this approach has been substantial across a wide variety of disciplines, notably having excellent compatibility with other techniques such as the previously mentioned graph and recurrent neural networks [10]. Some adaptation of this approach to disease modeling has already been attempted [35], but we expand on this idea with a novel approach to the training loss.…”

Physics-Informed Neural Network Method for Parabolic Differential Equations with Sharply Perturbed Initial Conditions

“…PINN is typically introduced as a method where the DNNs are trained using data and a penalty term in the form of the square of the PDE residuals evaluated at a set of residual points ( [11,4,3]). To facilitate this discussion, we consider the general PDE problem…”