Solving and learning nonlinear PDEs with Gaussian processes

Chen, Yifan; Hosseini, Bamdad; Owhadi, Houman; Stuart, Andrew M.

doi:10.1016/j.jcp.2021.110668

Cited by 90 publications

(71 citation statements)

References 112 publications

(218 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, due to technological and theoretical advances, there has been a renewed interest, especially for the case of multiscale/complex systems [68,42,41,69,70,71].…”

Section: Discussionmentioning

confidence: 99%

Learning black- and gray-box chemotactic PDEs/closures from agent based Monte Carlo simulation data

Lee¹,

Psarellis²,

Siettos³

et al. 2022

Preprint

View full text Add to dashboard Cite

We propose a machine learning framework for the data-driven discovery of macroscopic chemotactic Partial Differential Equations (PDEs) -and the closures that lead to themfrom high-fidelity, individual-based stochastic simulations of E.coli bacterial motility.The fine scale, detailed, hybrid (continuum -Monte Carlo) simulation model embodies the underlying biophysics, and its parameters are informed from experimental observations of individual cells. We exploit Automatic Relevance Determination (ARD) within a Gaussian Process framework for the identification of a parsimonious set of collective observables that parametrize the law of the effective PDEs. Using these observables, in a second step we learn effective, coarse-grained "Keller-Segel class" chemotactic PDEs using machine learning regressors: (a) (shallow) feedforward neural networks and (b) Gaussian Processes. The learned laws can be black-box (when no prior knowledge about the PDE law structure is assumed) or gray-box when parts of the equation (e.g. the pure diffusion part) is known and "hardwired" in the regression process. We also discuss data-driven corrections (both additive and functional) of analytically known, approximate closures.

show abstract

“…Recently, due to technological and theoretical advances, there has been a renewed interest, especially for the case of multiscale/complex systems [68,42,41,69,70,71].…”

Section: Discussionmentioning

confidence: 99%

Learning black- and gray-box chemotactic PDEs/closures from agent based Monte Carlo simulation data

Lee¹,

Psarellis²,

Siettos³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…A random variable for the cell densities from the Gaussian process, defined such that u = u * + σ n z or u * | X, u, X * ∼ N (ū * , Cov(u * )). X * ∈ R 2×nm (15) Similarly to X, except instead of (x 1 , . .…”

Section: Data Thresholdingmentioning

confidence: 99%

Computationally efficient mechanism discovery for cell invasion with uncertainty quantification

VandenHeuvel

Drovandi

Simpson

2022

Preprint

View full text Add to dashboard Cite

Parameter estimation for mathematical models of biological processes is often difficult and depends significantly on the quality and quantity of available data. We introduce an efficient framework using Gaussian processes to discover mechanisms underlying delay, migration, and proliferation in a cell invasion experiment. Gaussian processes are leveraged with bootstrapping to provide uncertainty quantification for the mechanisms that drive the invasion process. Our frame-work is efficient, parallelisable, and can be applied to other biological problems. We illustrate our methods using a canonical scratch assay experiment, demonstrating how simply we can explore different functional forms and develop and test hypotheses about underlying mechanisms, such as whether delay is present. All code and data to reproduce this work is available at https://github.com/DanielVandH/EquationLearning.jl.1Author summaryIn this work we introduce uncertainty quantification into equation learning methods, such as physics-informed and biologically-informed neural networks. Our framework is efficient and applicable to problems with unknown nonlinear mechanisms that we wish to learn from experiments where only sparse noisy data is available. We demonstrate our methods on a canonical scratch assay experiment from cell biology and show the underlying mechanisms can be learned, providing uncertainty intervals for functional forms and for solutions to partial differential equation models believed to describe the experiment.

show abstract

“…First, that of the solution of the inverse problem, i.e. that of identifying/discovering the hidden macroscopic laws, thus learning nonlinear operators and constructing coarse-scale dynamical models of ODEs and PDEs and their closures, from microscopic large-scale simulations and/or from multi-fidelity observations [10,57,58,59,62,9,3,47,74,15,16,48]. Second, based on the constructed coarse-scale models, to systematically investigate their dynamics by efficiently solving the corresponding differential equations, especially when dealing with (high-dimensional) PDEs [24,13,15,16,22,23,38,49,59,63].…”

Section: Introductionmentioning

confidence: 99%

“…that of identifying/discovering the hidden macroscopic laws, thus learning nonlinear operators and constructing coarse-scale dynamical models of ODEs and PDEs and their closures, from microscopic large-scale simulations and/or from multi-fidelity observations [10,57,58,59,62,9,3,47,74,15,16,48]. Second, based on the constructed coarse-scale models, to systematically investigate their dynamics by efficiently solving the corresponding differential equations, especially when dealing with (high-dimensional) PDEs [24,13,15,16,22,23,38,49,59,63]. Towards this aim, physics-informed machine learning [57,58,59,48,53,15,16,40] has been addressed to integrate available/incomplete information from the underlying physics, thus relaxing the "curse of dimensionality".…”

Section: Introductionmentioning

confidence: 99%

“…Second, based on the constructed coarse-scale models, to systematically investigate their dynamics by efficiently solving the corresponding differential equations, especially when dealing with (high-dimensional) PDEs [24,13,15,16,22,23,38,49,59,63]. Towards this aim, physics-informed machine learning [57,58,59,48,53,15,16,40] has been addressed to integrate available/incomplete information from the underlying physics, thus relaxing the "curse of dimensionality". However, failures may arise at the training phase especially in deep learning formulations, while there is still the issue of the corresponding computational cost [45,75,76,40].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Parsimonious Physics-Informed Random Projection Neural Networks for Initial-Value Problems of ODEs and index-1 DAEs

Fabiani¹,

Galaris²,

Russo³

et al. 2022

Preprint

View full text Add to dashboard Cite

We address a physics-informed neural network based on the concept of random projections for the numerical solution of initial value problems of nonlinear ODEs in linear-implicit form and index-1 DAEs, which may also arise from the spatial discretization of PDEs. The proposed scheme has a single hidden layer with appropriately randomly parametrized Gaussian kernels and a linear output layer, while the internal weights are fixed to ones. The unknown weights between the hidden and output layer are computed by Newton's iterations, using the Moore-Penrose pseudoinverse for low to medium scale, and sparse QR decomposition with L 2 regularization for medium to large scale systems. To deal with stiffness and sharp gradients, we thus propose an variable step size scheme based on the elementary local error control algorithm for adjusting the step size of integration and address a natural continuation method for providing good initial guesses for the Newton iterations. Building on previous works on random projections, we prove the approximation capability of the scheme for ODEs in the canonical form and index-1 DAEs in the semiexplicit form. The "optimal" bounds of the uniform distribution from which the values of the shape parameters of the Gaussian kernels are sampled are "parsimoniously" chosen based on the bias-variance trade-off decomposition, thus using the stiff van der Pol model as the reference solution for this task. The optimal bounds are fixed once and for all the problems studied here. In particular, the performance of the scheme is assessed through seven benchmark problems. Namely, we considered four index-1 DAEs, the Robertson model, a no autonomous model of five DAEs describing the motion of a bead on a rotating needle, a non autonomous model of six DAEs describing a power discharge control problem, the chemical so-called Akzo Nobel problem and three stiff problems, the Belousov-Zhabotinsky model, the Allen-Cahn PDE phase-field model and the Kuramoto-Sivashinsky PDE giving rise to chaotic dynamics. The efficiency of the scheme in terms of both numerical accuracy and computational cost is compared with three stiff/DAE solvers (ode23t, ode23s, ode15s) of the MATLAB ODE suite. Our results show that, the proposed scheme outperforms the aforementioned stiff solvers in several * Corresponding author A PREPRINT -MARCH 14, 2022 cases, especially in regimes where high stiffness and/or sharp gradients arise, in terms of numerical accuracy, while the computational costs are for any practical purposes comparable.

show abstract

Solving and learning nonlinear PDEs with Gaussian processes

Cited by 90 publications

References 112 publications

Learning black- and gray-box chemotactic PDEs/closures from agent based Monte Carlo simulation data

Learning black- and gray-box chemotactic PDEs/closures from agent based Monte Carlo simulation data

Computationally efficient mechanism discovery for cell invasion with uncertainty quantification

Parsimonious Physics-Informed Random Projection Neural Networks for Initial-Value Problems of ODEs and index-1 DAEs

Contact Info

Product

Resources

About