Numerical Bifurcation Analysis of PDEs From Lattice Boltzmann Model Simulations: a Parsimonious Machine Learning Approach

Abstract

We address a three-tier data-driven approach for the numerical solution of the inverse problem in Partial Differential Equations (PDEs) and for their numerical bifurcation analysis from spatio-temporal data produced by Lattice Boltzmann model simulations using machine learning. In the first step, we exploit manifold learning and in particular parsimonious Diffusion Maps using leave-one-out cross-validation (LOOCV) to both identify the intrinsic dimension of the manifold where the emergent dynamics evolve and for feature selection over the parameter space. In the second step, based on the selected features, we learn the right-hand-side of the effective PDEs using two machine learning schemes, namely shallow Feedforward Neural Networks (FNNs) with two hidden layers and single-layer Random Projection Networks (RPNNs), which basis functions are constructed using an appropriate random sampling approach. Finally, based on the learned black-box PDE model, we construct the corresponding bifurcation diagram, thus exploiting the numerical bifurcation analysis toolkit. For our illustrations, we implemented the proposed method to perform numerical bifurcation analysis of the 1D FitzHugh-Nagumo PDEs from data generated by D1Q3 Lattice Boltzmann simulations. The proposed method was quite effective in terms of numerical accuracy regarding the construction of the coarse-scale bifurcation diagram. Furthermore, the proposed RPNN scheme was $$\sim $$ ∼ 20 to 30 times less costly regarding the training phase than the traditional shallow FNNs, thus arising as a promising alternative to deep learning for the data-driven numerical solution of the inverse problem for high-dimensional PDEs.

Data-driven Control of Agent-based Models: an Equation/Variable-free Machine Learning Approach

“…Uncertainties range from the selection of an appropriate set of macroscopic variables for the description of the emergent behavior, to the closures that are required to bridge the micro-and macro-scales and allow the construction of reduced-order surrogate machine-learning models (see e.g. [12,13,20,56]) and/or learning the non-linear differential operators using for example DeepOnet [57] or Fourier Neural Operator for parametric PDEs [58]. For example, for the extraction of PDEs from data, challenges remain across algorithmic identification/learning steps pertaining to: (a) the "correct" structure of the evolution equation for the emergent dynamics (e.g., the inclusion or not of integral terms) [59], (b) higher-order spatial derivatives from strongly noisy microscopic simulations, (c) the correct coarse-grained boundary conditions for problems with non-trivial dynamics such as the ones of the stochastic agent-based model that we considered here, and/or (d) the type of the "color" of the noise when comes to the identification of SDEs.…”

“…Scientific computation and control of the emergent/collective dynamics of high-dimensional multiscale/complex dynamical systems constitute open challenging tasks due to (a) the lack of physical insight and knowledge of the appropriate macroscopic quantities needed to usefully describe the evolution of the emergent dynamics, (b) the so-called "curse of dimensionality" when trying to efficiently learn surrogate models with good generalization properties, and (c) the problem of bridging the scale where individual units (atoms, molecules, cells, bacteria, individuals, robots) interact, and the macroscopic scale where the emergent properties arise and evolve [1][2][3][4]. For the task of identification of macroscopic variables from high-fidelity simulations/spatio-temporal data, various machine learning methods have been proposed including non-linear manifold learning algorithms such as Diffusion Maps (DMs) [5][6][7][8][9][10][11][12][13], ISOMAP [14][15][16] and Local Linear Embedding [17,18] but also Autoencoders [19,20]. For the task of the extraction of surrogate models for the approximation of the emergent dynamics, available approaches include the Sparse Identification of the Nonlinear Dynamics (SINDy) [21], the Koopman operator [22][23][24][25][26][27], Gaussian Processes [12,18,28], Artificial Neural Networks (ANNs) [12,13], Recursive Neural Networks (RNN) [20], Deep Learning [29], as well as Long Short-Term Memory (LSTM) networks [30].…”

“…For the task of identification of macroscopic variables from high-fidelity simulations/spatio-temporal data, various machine learning methods have been proposed including non-linear manifold learning algorithms such as Diffusion Maps (DMs) [5][6][7][8][9][10][11][12][13], ISOMAP [14][15][16] and Local Linear Embedding [17,18] but also Autoencoders [19,20]. For the task of the extraction of surrogate models for the approximation of the emergent dynamics, available approaches include the Sparse Identification of the Nonlinear Dynamics (SINDy) [21], the Koopman operator [22][23][24][25][26][27], Gaussian Processes [12,18,28], Artificial Neural Networks (ANNs) [12,13], Recursive Neural Networks (RNN) [20], Deep Learning [29], as well as Long Short-Term Memory (LSTM) networks [30]. For the task of the bridging of the micro-and macro-scales, the Equation-free (EF) multiscale framework, introduced back in the early 2000s years, [1,31,32] bypasses the need to extract explicit, closed form macroscopic/surrogate models of any type (e.g.…”

Physics-agnostic and physics-infused machine learning for thin films flows: modelling, and predictions from small data

“…2020; Galaris et al. 2022), among others. Extensions of PDE identification including grey box or closure identification (such as those explored in our work) have been studied in the context of various applications (Parish & Duraisamy 2016; Pan & Duraisamy 2018; Duraisamy, Iaccarino & Xiao 2019; Lee et al.…”

Physics-agnostic and Physics-infused machine learning for thin films flows: modeling, and predictions from small data

“…Our work falls in the category of dynamical system identification [5][6][7][8]. Recently increased interest in PDE identification has led to the development of alternative algorithmic tools, such as sparse identification of nonlinear dynamical systems using dictionaries [9,10], PDE-net [11], physics-informed neural networks [12], and others [13][14][15] Our algorithmic approach can be implemented on data from detailed PDE simulations [16], agent-based modeling [17,18] or Lattice Boltzmann simulations [19,20] among others. Extensions of PDE identification including gray-box or closure identification (such as those explored in our work) have been studied in the context of various applications [16,17,[21][22][23][24][25][26].…”