Extreme learning machine collocation for the numerical solution of elliptic PDEs with sharp gradients

Calabrò, Francesco; Fabiani, Gianluca; Siettos, Constantinos

doi:10.1016/j.cma.2021.114188

Cited by 45 publications

(21 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although it has been demonstrated that neural networks are universal function approximators, the proper choice of architecture generally aids learning and certain challenging problems (e.g. solving non-linear PDEs) may require more specific architectures to capture all their properties (see for instance Extreme Learning Machine 61 , 62 ). For that reason, we have proposed a new architecture, inspired by 51 , to solve non-linear PDEs with discontinuities under two assumptions.…”

Section: Piann Architecturementioning

confidence: 99%

Physics-informed attention-based neural network for hyperbolic partial differential equations: application to the Buckley–Leverett problem

Rodriguez-Torrado

Ruiz²,

Cueto‐Felgueroso

et al. 2022

Sci Rep

View full text Add to dashboard Cite

Physics-informed neural networks (PINNs) have enabled significant improvements in modelling physical processes described by partial differential equations (PDEs) and are in principle capable of modeling a large variety of differential equations. PINNs are based on simple architectures, and learn the behavior of complex physical systems by optimizing the network parameters to minimize the residual of the underlying PDE. Current network architectures share some of the limitations of classical numerical discretization schemes when applied to non-linear differential equations in continuum mechanics. A paradigmatic example is the solution of hyperbolic conservation laws that develop highly localized nonlinear shock waves. Learning solutions of PDEs with dominant hyperbolic character is a challenge for current PINN approaches, which rely, like most grid-based numerical schemes, on adding artificial dissipation. Here, we address the fundamental question of which network architectures are best suited to learn the complex behavior of non-linear PDEs. We focus on network architecture rather than on residual regularization. Our new methodology, called physics-informed attention-based neural networks (PIANNs), is a combination of recurrent neural networks and attention mechanisms. The attention mechanism adapts the behavior of the deep neural network to the non-linear features of the solution, and break the current limitations of PINNs. We find that PIANNs effectively capture the shock front in a hyperbolic model problem, and are capable of providing high-quality solutions inside the convex hull of the training set.

show abstract

Section: Piann Architecturementioning

confidence: 99%

Physics-informed attention-based neural network for hyperbolic partial differential equations: application to the Buckley–Leverett problem

Rodriguez-Torrado

Ruiz²,

Cueto‐Felgueroso

et al. 2022

Sci Rep

View full text Add to dashboard Cite

show abstract

“…that of identifying/discovering the hidden macroscopic laws, thus learning nonlinear operators and constructing coarse-scale dynamical models of ODEs and PDEs and their closures, from microscopic large-scale simulations and/or from multi-fidelity observations [10,57,58,59,62,9,3,47,74,15,16,48]. Second, based on the constructed coarse-scale models, to systematically investigate their dynamics by efficiently solving the corresponding differential equations, especially when dealing with (high-dimensional) PDEs [24,13,15,16,22,23,38,49,59,63]. Towards this aim, physics-informed machine learning [57,58,59,48,53,15,16,40] has been addressed to integrate available/incomplete information from the underlying physics, thus relaxing the "curse of dimensionality".…”

Section: Introductionmentioning

confidence: 99%

Parsimonious Physics-Informed Random Projection Neural Networks for Initial-Value Problems of ODEs and index-1 DAEs

Fabiani¹,

Galaris²,

Russo³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

We address a physics-informed neural network based on the concept of random projections for the numerical solution of initial value problems of nonlinear ODEs in linear-implicit form and index-1 DAEs, which may also arise from the spatial discretization of PDEs. The proposed scheme has a single hidden layer with appropriately randomly parametrized Gaussian kernels and a linear output layer, while the internal weights are fixed to ones. The unknown weights between the hidden and output layer are computed by Newton's iterations, using the Moore-Penrose pseudoinverse for low to medium scale, and sparse QR decomposition with L 2 regularization for medium to large scale systems. To deal with stiffness and sharp gradients, we thus propose an variable step size scheme based on the elementary local error control algorithm for adjusting the step size of integration and address a natural continuation method for providing good initial guesses for the Newton iterations. Building on previous works on random projections, we prove the approximation capability of the scheme for ODEs in the canonical form and index-1 DAEs in the semiexplicit form. The "optimal" bounds of the uniform distribution from which the values of the shape parameters of the Gaussian kernels are sampled are "parsimoniously" chosen based on the bias-variance trade-off decomposition, thus using the stiff van der Pol model as the reference solution for this task. The optimal bounds are fixed once and for all the problems studied here. In particular, the performance of the scheme is assessed through seven benchmark problems. Namely, we considered four index-1 DAEs, the Robertson model, a no autonomous model of five DAEs describing the motion of a bead on a rotating needle, a non autonomous model of six DAEs describing a power discharge control problem, the chemical so-called Akzo Nobel problem and three stiff problems, the Belousov-Zhabotinsky model, the Allen-Cahn PDE phase-field model and the Kuramoto-Sivashinsky PDE giving rise to chaotic dynamics. The efficiency of the scheme in terms of both numerical accuracy and computational cost is compared with three stiff/DAE solvers (ode23t, ode23s, ode15s) of the MATLAB ODE suite. Our results show that, the proposed scheme outperforms the aforementioned stiff solvers in several * Corresponding author A PREPRINT -MARCH 14, 2022 cases, especially in regimes where high stiffness and/or sharp gradients arise, in terms of numerical accuracy, while the computational costs are for any practical purposes comparable.

show abstract

“…A number of further developments of the ELM technique for solving linear and nonlinear PDEs appeared recently; see e.g. [10,6,19,12,17], among others. In order to address the influence of random initialization of the hidden-layer coefficients on the ELM accuracy, a modified batch intrinsic plasticity (modBIP) method is developed in [10] for pre-training the random coefficients in the ELM network.…”

Section: Introductionmentioning

confidence: 99%

“…The accuracy of the combined modBIP/ELM method has been shown to be insensitive to the random initializations of the hidden-layer coefficients. In [6] the authors present a method for solving one-dimensional linear elliptic PDEs based on ELM with single hidden-layer feedforward neural networks and the sigmoid activation function. The random parameters in the activation function are set based on the location of the domain of interest and the function derivative information.…”

Section: Introductionmentioning

confidence: 99%

Numerical Computation of Partial Differential Equations by Hidden-Layer Concatenated Extreme Learning Machine

Ni¹,

Dong²

2022

Preprint

View full text Add to dashboard Cite

The extreme learning machine (ELM) method can yield highly accurate solutions to linear/nonlinear partial differential equations (PDEs), but requires the last hidden layer of the neural network to be wide to achieve a high accuracy. If the last hidden layer is narrow, the accuracy of the existing ELM method will be poor, irrespective of the rest of the network configuration. In this paper we present a modified ELM method, termed HLConcELM (hidden-layer concatenated ELM), to overcome the above drawback of the conventional ELM method. The HLConcELM method can produce highly accurate solutions to linear/nonlinear PDEs when the last hidden layer of the network is narrow and when it is wide. The new method is based on a type of modified feedforward neural networks (FNN), termed HLConcFNN (hidden-layer concatenated FNN), which incorporates a logical concatenation of the hidden layers in the network and exposes all the hidden nodes to the output-layer nodes. We show that HLConcFNNs have the remarkable property that, given a network architecture, when additional hidden layers are appended to the network or when extra nodes are added to the existing hidden layers, the approximation capacity of the HLConcFNN associated with the new architecture is guaranteed to be not smaller than that of the original network architecture. We present ample benchmark tests with linear/nonlinear PDEs to demonstrate the computational accuracy and performance of the HLConcELM method and the superiority of this method to the conventional ELM from previous works.

show abstract

Extreme learning machine collocation for the numerical solution of elliptic PDEs with sharp gradients

Cited by 45 publications

References 39 publications

Physics-informed attention-based neural network for hyperbolic partial differential equations: application to the Buckley–Leverett problem

Physics-informed attention-based neural network for hyperbolic partial differential equations: application to the Buckley–Leverett problem

Parsimonious Physics-Informed Random Projection Neural Networks for Initial-Value Problems of ODEs and index-1 DAEs

Numerical Computation of Partial Differential Equations by Hidden-Layer Concatenated Extreme Learning Machine

Contact Info

Product

Resources

About