Transfer learning for deep neural network-based partial differential equations solving

Chen, Xinhai; Gong, Chunye; Wan, Qian; Deng, Liang; Wan, Yunbo; Liu, Yang; Chen, Bin; Liu, Jie

doi:10.1186/s42774-021-00094-7

Cited by 26 publications

(17 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The reduction in the amount of training time is substantial, especially when the Reynolds numbers in the targeted DNN and base DNN are close. This observation is consistent with the finding reported by Chen et al [36].…”

Section: Transfer Learningsupporting

confidence: 94%

See 1 more Smart Citation

A Direct-Forcing Immersed Boundary Method for Incompressible Flows Based on Physics-Informed Neural Network

Huang

Zhang

2022

Fluids

View full text Add to dashboard Cite

The application of physics-informed neural networks (PINNs) to computational fluid dynamics simulations has recently attracted tremendous attention. In the simulations of PINNs, the collocation points are required to conform to the fluid–solid interface on which no-slip boundary condition is enforced. Here, a novel PINN that incorporates the direct-forcing immersed boundary (IB) method is developed. In the proposed IB-PINN, the boundary conforming requirement in arranging the collocation points is eliminated. Instead, velocity penalties at some marker points are added to the loss function to enforce no-slip condition at the fluid–solid interface. In addition, force penalties at some collocation points are also added to the loss function to ensure compact distribution of the volume force. The effectiveness of IB-PINN in solving incompressible Navier–Stokes equations is demonstrated through the simulation of laminar flow past a circular cylinder that is placed in a channel. The solution obtained using the IB-PINN is compared with two reference solutions obtained using a conventional mesh-based IB method and an ordinary body-fitted grid method. The comparison indicates that the three solutions are in excellent agreement with each other. The influences of some parameters, such as weights for different loss components, numbers of collocation and marker points, hyperparameters in the neural network, etc., on the performance of IB-PINN are also studied. In addition, a transfer learning experiment is conducted on solving Navier–Stokes equations with different Reynolds numbers.

show abstract

Section: Transfer Learningsupporting

confidence: 94%

“…This technique has been successfully applied to the fields of computer vision and natural language processing. In one recent work by Chen et al [36], it was shown that for solving N-S equations at different Reynolds numbers using PINN, the re-training benefited from the transfer learning technique.…”

Section: Transfer Learningmentioning

confidence: 99%

A Direct-Forcing Immersed Boundary Method for Incompressible Flows Based on Physics-Informed Neural Network

Huang

Zhang

2022

Fluids

View full text Add to dashboard Cite

show abstract

“…Transfer learning. One limitation of the network-based method to solve PDEs is the slow training speed, and there is some work to improve this, such as the transfer learning [8] and manifold learning [24]. For the Boltzmann equation, especially for the BGK model, the classical methods such as 3) The relative error between the numerical solution by NR/NSR and the reference solution for the density ρ, macroscopic velocity u 1 and the temperature T with Kn = 0.01, 0.1 and 1 at t = 0 and 0.1.…”

Section: Algorithm 51 Obtain the Data-driven Collision Kernelmentioning

confidence: 99%

Solving Boltzmann equation with neural sparse representation

Li¹,

Wang²,

Liu³

et al. 2023

Preprint

View full text Add to dashboard Cite

We consider the neural sparse representation to solve Boltzmann equation with BGK and quadratic collision model, where a network-based ansatz that can approximate the distribution function with extremely high efficiency is proposed. Precisely, fully connected neural networks are employed in the time and spatial space so as to avoid the discretization in space and time. The different low-rank representations are utilized in the microscopic velocity for the BGK and quadratic collision model, resulting in a significant reduction in the degree of freedom. We approximate the discrete velocity distribution in the BGK model using the canonical polyadic decomposition. For the quadratic collision model, a data-driven, SVD-based linear basis is built based on the BGK solution. All these will significantly improve the efficiency of the network when solving Boltzmann equation. Moreover, the specially designed adaptive-weight loss function is proposed with the strategies as multi-scale input and Maxwellian splitting applied to further enhance the approximation efficiency and speed up the learning process. Several numerical experiments, including 1D wave and Sod problems and 2D wave problem, demonstrate the effectiveness of these neural sparse representation methods.

show abstract

“…In addition, the grid search algorithm and early stopping are utilized to find architectures that achieve satisfying performance on specific physical problems within a short time period. By using this framework, accurate prediction results can be obtained, and at the same time transfer learning [28], [30] is utilized to increase the scalability of the trained model. By utilizing the similarities between equations, it can greatly improve the training efficiency for solving different problems.…”

Section: Introductionmentioning

confidence: 99%

AutoKE: An Automatic Knowledge Embedding Framework for Scientific Machine Learning

Chen

Zhang

2023

IEEE Trans. Artif. Intell.

View full text Add to dashboard Cite

Imposing physical constraints on neural networks as a method of knowledge embedding has achieved great progress in solving physical problems described by governing equations. However, for many engineering problems, governing equations often have complex forms, including complex partial derivatives or stochastic physical fields, which results in significant inconveniences from the perspective of implementation. In this paper, to effectively automate the process of embedding physical knowledge, a scientific machine learning framework, called AutoKE, is proposed, and a reservoir flow problem with a complex governing equation is taken as an instance. In AutoKE, an emulator comprised of deep neural networks (DNNs) is built for predicting the physical variables of interest. An arbitrarily complex equation can be parsed and automatically converted into a computational graph through the equation parser module, and the fitness of the emulator to the governing equation is evaluated via automatic differentiation. Furthermore, the fixed weights in the loss function are substituted with adaptive weights by incorporating the Lagrangian dual method. Neural architecture search (NAS) is also introduced into the AutoKE to select an optimal network architecture of the emulator according to the specific problem. Finally, we apply transfer learning to enhance the scalability of the emulator. In experiments, the framework is verified by a series of physical problems in which it can automatically embed physical knowledge into an emulator without heavy hand-coding. The results demonstrate that the emulator can not only make accurate predictions, but also be applied to similar problems with high efficiency via transfer learning.Impact Statement -Embedding physical knowledge into machine learning has been widely applied in solving scientific computing problems. However, it is tedious and timeconsuming to establish the emulator and embed physical knowledge into it. To this end, we proposed a plug-and-play framework for solving complicated physical problems. The user inputs equations with arbitrary form according to the specified rules, which can be automatically parsed and transformed into a structural representation that the program can recognize. The embedding of physical knowledge, the establishment of emulators, and the adjustment of parameters are automated, which provides great convenience while obtaining accurate solutions. Moreover, the strong scalability enables the trained emulator to be applied to similar problems,

show abstract

Transfer learning for deep neural network-based partial differential equations solving

Cited by 26 publications

References 20 publications

A Direct-Forcing Immersed Boundary Method for Incompressible Flows Based on Physics-Informed Neural Network

A Direct-Forcing Immersed Boundary Method for Incompressible Flows Based on Physics-Informed Neural Network

Solving Boltzmann equation with neural sparse representation

AutoKE: An Automatic Knowledge Embedding Framework for Scientific Machine Learning

Contact Info

Product

Resources

About