Numerical solution and bifurcation analysis of nonlinear partial differential equations with extreme learning machines

Fabiani, Gianluca; Calabrò, Francesco; Russo, Lucia; Siettos, Constantinos

doi:10.1007/s10915-021-01650-5

Cited by 46 publications

(17 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Based on that configuration, the output is projected linearly onto the functional subspace spanned by the nonlinear basis functions of the hidden layer, and the only remaining unknowns are the weights between the hidden and the output layer. Their estimation is done by solving a nonlinear regularized least squares problem [60,61]. The universal approximation properties of the RPNNs has been proved in a series of papers (see e.g.…”

Section: Random Projection Neural Networkmentioning

confidence: 99%

Numerical Bifurcation Analysis of PDEs From Lattice Boltzmann Model Simulations: a Parsimonious Machine Learning Approach

et al. 2022

Self Cite

View full text Add to dashboard Cite

We address a three-tier data-driven approach for the numerical solution of the inverse problem in Partial Differential Equations (PDEs) and for their numerical bifurcation analysis from spatio-temporal data produced by Lattice Boltzmann model simulations using machine learning. In the first step, we exploit manifold learning and in particular parsimonious Diffusion Maps using leave-one-out cross-validation (LOOCV) to both identify the intrinsic dimension of the manifold where the emergent dynamics evolve and for feature selection over the parameter space. In the second step, based on the selected features, we learn the right-hand-side of the effective PDEs using two machine learning schemes, namely shallow Feedforward Neural Networks (FNNs) with two hidden layers and single-layer Random Projection Networks (RPNNs), which basis functions are constructed using an appropriate random sampling approach. Finally, based on the learned black-box PDE model, we construct the corresponding bifurcation diagram, thus exploiting the numerical bifurcation analysis toolkit. For our illustrations, we implemented the proposed method to perform numerical bifurcation analysis of the 1D FitzHugh-Nagumo PDEs from data generated by D1Q3 Lattice Boltzmann simulations. The proposed method was quite effective in terms of numerical accuracy regarding the construction of the coarse-scale bifurcation diagram. Furthermore, the proposed RPNN scheme was $$\sim $$ ∼ 20 to 30 times less costly regarding the training phase than the traditional shallow FNNs, thus arising as a promising alternative to deep learning for the data-driven numerical solution of the inverse problem for high-dimensional PDEs.

show abstract

Section: Random Projection Neural Networkmentioning

confidence: 99%

Numerical Bifurcation Analysis of PDEs From Lattice Boltzmann Model Simulations: a Parsimonious Machine Learning Approach

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…The nonlinear least squares method requires two procedures for solving the system (17), one for computing the residual of this system and the other for computing the Jacobian matrix for a given arbitrary…”

Section: Solving Linear/nonlinear Pdes With Hidden-layer Concatenated...mentioning

confidence: 99%

“…A number of further developments of the ELM technique for solving linear and nonlinear PDEs appeared recently; see e.g. [10,6,19,12,17], among others. In order to address the influence of random initialization of the hidden-layer coefficients on the ELM accuracy, a modified batch intrinsic plasticity (modBIP) method is developed in [10] for pre-training the random coefficients in the ELM network.…”

Section: Introductionmentioning

confidence: 99%

“…Here, by "outperform" we mean that one method achieves a better accuracy under the same computational cost or incurs a lower computational cost to achieve the same accuracy. In [17] an ELM method is presented for the numerical solution of stationary nonlinear PDEs based on the sigmoid and radial basis activation functions. The authors observe that the ELM method exhibits a better accuracy than the finite difference and the finite element method.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Numerical Computation of Partial Differential Equations by Hidden-Layer Concatenated Extreme Learning Machine

Ni¹,

Dong²

2022

Preprint

View full text Add to dashboard Cite

The extreme learning machine (ELM) method can yield highly accurate solutions to linear/nonlinear partial differential equations (PDEs), but requires the last hidden layer of the neural network to be wide to achieve a high accuracy. If the last hidden layer is narrow, the accuracy of the existing ELM method will be poor, irrespective of the rest of the network configuration. In this paper we present a modified ELM method, termed HLConcELM (hidden-layer concatenated ELM), to overcome the above drawback of the conventional ELM method. The HLConcELM method can produce highly accurate solutions to linear/nonlinear PDEs when the last hidden layer of the network is narrow and when it is wide. The new method is based on a type of modified feedforward neural networks (FNN), termed HLConcFNN (hidden-layer concatenated FNN), which incorporates a logical concatenation of the hidden layers in the network and exposes all the hidden nodes to the output-layer nodes. We show that HLConcFNNs have the remarkable property that, given a network architecture, when additional hidden layers are appended to the network or when extra nodes are added to the existing hidden layers, the approximation capacity of the HLConcFNN associated with the new architecture is guaranteed to be not smaller than that of the original network architecture. We present ample benchmark tests with linear/nonlinear PDEs to demonstrate the computational accuracy and performance of the HLConcELM method and the superiority of this method to the conventional ELM from previous works.

show abstract

“…that of identifying/discovering the hidden macroscopic laws, thus learning nonlinear operators and constructing coarse-scale dynamical models of ODEs and PDEs and their closures, from microscopic large-scale simulations and/or from multi-fidelity observations [10,57,58,59,62,9,3,47,74,15,16,48]. Second, based on the constructed coarse-scale models, to systematically investigate their dynamics by efficiently solving the corresponding differential equations, especially when dealing with (high-dimensional) PDEs [24,13,15,16,22,23,38,49,59,63]. Towards this aim, physics-informed machine learning [57,58,59,48,53,15,16,40] has been addressed to integrate available/incomplete information from the underlying physics, thus relaxing the "curse of dimensionality".…”

Section: Introductionmentioning

confidence: 99%

Parsimonious Physics-Informed Random Projection Neural Networks for Initial-Value Problems of ODEs and index-1 DAEs

Fabiani¹,

Galaris²,

Russo³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

We address a physics-informed neural network based on the concept of random projections for the numerical solution of initial value problems of nonlinear ODEs in linear-implicit form and index-1 DAEs, which may also arise from the spatial discretization of PDEs. The proposed scheme has a single hidden layer with appropriately randomly parametrized Gaussian kernels and a linear output layer, while the internal weights are fixed to ones. The unknown weights between the hidden and output layer are computed by Newton's iterations, using the Moore-Penrose pseudoinverse for low to medium scale, and sparse QR decomposition with L 2 regularization for medium to large scale systems. To deal with stiffness and sharp gradients, we thus propose an variable step size scheme based on the elementary local error control algorithm for adjusting the step size of integration and address a natural continuation method for providing good initial guesses for the Newton iterations. Building on previous works on random projections, we prove the approximation capability of the scheme for ODEs in the canonical form and index-1 DAEs in the semiexplicit form. The "optimal" bounds of the uniform distribution from which the values of the shape parameters of the Gaussian kernels are sampled are "parsimoniously" chosen based on the bias-variance trade-off decomposition, thus using the stiff van der Pol model as the reference solution for this task. The optimal bounds are fixed once and for all the problems studied here. In particular, the performance of the scheme is assessed through seven benchmark problems. Namely, we considered four index-1 DAEs, the Robertson model, a no autonomous model of five DAEs describing the motion of a bead on a rotating needle, a non autonomous model of six DAEs describing a power discharge control problem, the chemical so-called Akzo Nobel problem and three stiff problems, the Belousov-Zhabotinsky model, the Allen-Cahn PDE phase-field model and the Kuramoto-Sivashinsky PDE giving rise to chaotic dynamics. The efficiency of the scheme in terms of both numerical accuracy and computational cost is compared with three stiff/DAE solvers (ode23t, ode23s, ode15s) of the MATLAB ODE suite. Our results show that, the proposed scheme outperforms the aforementioned stiff solvers in several * Corresponding author A PREPRINT -MARCH 14, 2022 cases, especially in regimes where high stiffness and/or sharp gradients arise, in terms of numerical accuracy, while the computational costs are for any practical purposes comparable.

show abstract

Numerical solution and bifurcation analysis of nonlinear partial differential equations with extreme learning machines

Cited by 46 publications

References 56 publications

Numerical Bifurcation Analysis of PDEs From Lattice Boltzmann Model Simulations: a Parsimonious Machine Learning Approach

Numerical Bifurcation Analysis of PDEs From Lattice Boltzmann Model Simulations: a Parsimonious Machine Learning Approach

Numerical Computation of Partial Differential Equations by Hidden-Layer Concatenated Extreme Learning Machine

Parsimonious Physics-Informed Random Projection Neural Networks for Initial-Value Problems of ODEs and index-1 DAEs

Contact Info

Product

Resources

About