Multi-Fidelity Physics-Constrained Neural Network and Its Application in Materials Modeling

Liu, Dehao; Wang, Yan

doi:10.1115/1.4044400

Cited by 135 publications

(43 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We characterize the phenomenon of vanishing task-specific gradients, demonstrate its impact on the accuracy of a learned function approximation, and empirically validate Proposition 2.2.1. For this, we consider a generic neural network with five layers and 64 neurons per layer, tasked with learning a 2D function that contains a wide spectrum of frequencies and unknown coefficients ξ = ( ξ1 , ξ2 , ξ3 , ξ4 ) using Sobolev training with the loss from equation (10). The details of the test problem and the training setup are given in appendix B.…”

Section: Inverse Dirichlet Weighting Avoids Vanishing Task-specific G...mentioning

confidence: 99%

“…The popular physics informed neural networks (PINNs) rely on knowing a differential equation model of the system in order to solve a soft-constrained optimization problem [5,8]. Thanks to their mesh-free character, PINNs can be used for both forward and inverse modeling in domains including material science [9][10][11], fluid dynamics [8,[12][13][14] and turbulence [15,16], biology [17], medicine [18,19],…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Inverse Dirichlet weighting enables reliable training of physics informed neural networks

Maddu

Sturm

Müller

et al. 2022

Mach. Learn.: Sci. Technol.

View full text Add to dashboard Cite

We characterize and remedy a failure mode that may arise from multi-scale dynamics with scale imbalances during training of deep neural networks, such as Physics Informed Neural Networks (PINNs). PINNs are popular machine-learning templates that allow for seamless integration of physical equation models with data. Their training amounts to solving an optimization problem over a weighted sum of data-fidelity and equation-fidelity objectives. Conflicts between objectives can arise from scale imbalances, heteroscedasticity in the data, stiffness of the physical equation, or from catastrophic interference during sequential training. We explain the training pathology arising from this and propose a simple yet effective inverse Dirichlet weighting strategy to alleviate the issue. We compare with Sobolev training of neural networks, providing the baseline of analytically ε-optimal training. We demonstrate the effectiveness of inverse Dirichlet weighting in various applications, including a multi-scale model of active turbulence, where we show orders of magnitude improvement in accuracy and convergence over conventional PINN training. For inverse modeling using sequential training, we find that inverse Dirichlet weighting protects a PINN against catastrophic forgetting.

show abstract

Section: Inverse Dirichlet Weighting Avoids Vanishing Task-specific G...mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Inverse Dirichlet weighting enables reliable training of physics informed neural networks

Maddu

Sturm

Müller

et al. 2022

Mach. Learn.: Sci. Technol.

View full text Add to dashboard Cite

show abstract

“…Physics-informed neural networks have been shown to produce good results for some applications, such as the modelling of materials [26] and high-speed flows [27], but show limitations in their current form that are not yet well understood [28]. For instance, it has been demonstrated that physics-informed models fail to provide good solution approximations for flow in porous media when shocks are introduced [29] and that some LSTM models may not perfectly resolve turbulent conditions [30].…”

Section: Image Super-resolution Applied To Cfdmentioning

confidence: 99%

Using Physics-Informed Generative Adversarial Networks to Perform Super-Resolution for Multiphase Fluid Simulations

Li¹,

McComb²

2022

Preprint

View full text Add to dashboard Cite

Computational Fluid Dynamics (CFD) simulations are useful to the field of engineering design as they provide deep insights on product or system performance without the need to construct and test physical prototypes. However, they can be very computationally intensive to run. Machine learning methods have been shown to reconstruct high- resolution single-phase turbulent fluid flow simulations from low-resolution inputs. This offers a potential avenue towards alleviating computational cost in iterative engineering design applications. However, little work thus far has explored the application of machine learning image super-resolution methods to multiphase fluid flow (which is important for emerging fields such as marine hydrokinetic energy conversion). In this work, we apply a modified version of the Super-Resolution Generative Adversarial Network (SRGAN) model to a multiphase turbulent fluid flow problem, specifically to reconstruct fluid phase fraction at a higher resolution. Two models were created in this work, one which incorporates a physics-informed term in the loss function and one which does not, and the results are discussed and compared. We found that both models significantly outperform non-machine learning upsampling methods and can preserve a substantial amount of detail, showing the versatility of the SRGAN model for upsampling multi-phase fluid simulations. However, the difference in accuracy between the two models is minimal indicating that, in the context studied here, the additional complexity of a physics- informed approach may not be justified.

show abstract

“…Physics has also been used as a prior in reinforcement learning [222]. As low fidelity MB's capture the general trend and the high fidelity MBs additionally capture intricate local details, a multi-fidelity physics constrained NN was proposed for incorporating linear and nonlinear PDEs [223].…”

Section: Physics Based Regularizationmentioning

confidence: 99%

Driven by Data or Derived Through Physics? A Review of Hybrid Physics Guided Machine Learning Techniques With Cyber-Physical System (CPS) Focus

2020

View full text Add to dashboard Cite

A multitude of cyber-physical system (CPS) applications, including design, control, diagnosis, prognostics, and a host of other problems, are predicated on the assumption of model availability. There are mainly two approaches to modeling: Physics/Equation based modeling (Model-Based, MB) and Machine Learning (ML). Recently, there is a growing consensus that ML methodologies relying on data need to be coupled with prior scientific knowledge (or physics, MB) for modeling CPS. We refer to the paradigm that combines MB approaches with ML as hybrid learning methods. Hybrid modeling (HB) methods is a growing field within both the ML and scientific communities, and are recognized as an important emerging but nascent area of research. Recently, several works have attempted to merge MB and ML models for the complete exploitation of their combined potential. However, the research literature is scattered and unorganized. So, we make a meticulous and systematic attempt at organizing and standardizing the methods of combining ML and MB models. In addition to that, we outline five metrics for the comprehensive evaluation of hybrid models. Finally, we conclude by shedding some light on the challenges of hybrid models, which we, as a research community, should focus on for harnessing the full potential of hybrid models. An additional feature of this survey is that the hybrid modeling work has been discussed with a focus on modeling cyber-physical systems. INDEX TERMS Cyber-physical systems, deep learning, deep neural networks, hybrid models, model-based, machine learning, physics guided, physics informed, physics prior, theory guided.

show abstract

Multi-Fidelity Physics-Constrained Neural Network and Its Application in Materials Modeling

Cited by 135 publications

References 32 publications

Inverse Dirichlet weighting enables reliable training of physics informed neural networks

Inverse Dirichlet weighting enables reliable training of physics informed neural networks

Using Physics-Informed Generative Adversarial Networks to Perform Super-Resolution for Multiphase Fluid Simulations

Driven by Data or Derived Through Physics? A Review of Hybrid Physics Guided Machine Learning Techniques With Cyber-Physical System (CPS) Focus

Contact Info

Product

Resources

About