Predicting parametric spatiotemporal dynamics by multi-resolution PDE structure-preserved deep learning

Abstract

Although recent advances in deep learning (DL) have shown a great promise for learning physics exhibiting complex spatiotemporal dynamics, the high training cost, unsatisfying extrapolability for long-term predictions, and poor generalizability in out-of-sample regimes significantly limit their applications in science/engineering problems. A more promising way is to leverage available physical prior and domain knowledge to develop scientific DL models, known as physics-informed deep learning (PiDL). In most existing PiDL frameworks, e.g., physicsinformed neural networks, the physics prior is mainly utilized to regularize neural network training by incorporating governing equations into the loss function in a soft manner. In this work, we propose a new direction to leverage physics prior knowledge by baking the mathematical structures of governing equations into the neural network architecture design. In particular, we develop a novel PDE-preserved neural network (PPNN) for rapidly predicting parametric spatiotemporal dynamics, given the governing PDEs are (partially) known. The discretized PDE structures are preserved in PPNN as convolutional residual network (ConvResNet) blocks, which are formulated in a multi-resolution setting. This physics-inspired learning architecture design endows PPNN with excellent generalizability and long-term prediction accuracy compared to the state-of-the-art blackbox ConvResNet baseline. The effectiveness and merit of the proposed methods have been demonstrated over a handful of spatiotemporal dynamical

Updates on Model Hierarchies for Understanding and Simulating the Climate System: A Focus on Data‐Informed Methods and Climate Change Impacts

“…Climate modeling has already adopted numerous ideas from the field of AI and has, within a short period of time, witnessed a meteoric rise in ML-driven modeling (Reichstein et al, 2019). As discussed at the workshop, ML-assisted analyses have begun to pervade practically all aspects of the existing model hierarchy: from modeling fundamental partial differential equations (PDEs) and dynamical systems (Liu et al, 2022;Pathak et al, 2018a), to modeling and performing equation discovery for SGS processes (e.g., Brenowitz & Bretherton, 2019;Gentine et al, 2018;Rasp et al, 2018;Yuval & O'Gorman, 2020;Zanna & Bolton, 2020), to full-blown efforts to completely replace complex weather prediction models with a single ML model (Bi et al, 2022;Lam et al, 2022;Pathak et al, 2022). Moreover, rather than just being used to build new models, ML is also helping modelers improve existing models by aiding calibration and UQ, by providing emulators that approximate computationally expensive models, and by catalyzing the development of a new-class of data-driven parameterizations (e.g., Schneider et al, 2023).…”

“…NN-based prediction models can now generate increasingly precise predictions of simple chaotic systems, such as those modeled by higher-order nonlinear PDEs (Pathak et al, 2018a). This numerical capability of NNs has been generalized by Liu et al (2022) to develop faster deep learning numerical solvers that can replace traditional time-stepping numerical schemes, such as those heavily employed by climate models to solve the equations of motion. Rigorous tests on more advanced chaotic systems such as the Lorenz-96 model (L96) have reaffirmed the advantages of NN-based predictions (Chattopadhyay et al, 2020).…”

A Review of Physics-Informed Machine Learning in Fluid Mechanics

“…The utility of PINNs, along with its numerous variants, has been investigated in numerous studies [22,[26][27][28][29][30][31][32][33][34][35], on both spatiotemporal and inverse modeling. Other PIML methods for spatiotemporal modeling include the PDE-preserving NN (PPNN) [36], physics-informed convolution recurrent network (PhyCRNet) [37], and several others [38][39][40][41][42][43][44]. Some of these methods will be discussed later in Section 2.…”

“…Wang et al [87] proposed the equivariant NN that introduced symmetries into the ML model predictions. Liu et al [36] introduced a PDEpreserving NN (PPNN) architecture that demonstrated excellent stability for spatiotemporal predictions. The PPNN works by forming residual connections between input velocities downsampled on the right-hand side (RHS) of the Navier-Stokes equations to result in more stable predictions than a vanilla CNN model.…”

“…As opposed to PINN, where the physical constraints are embedded in the form of a loss function, in PPNN, the PDE structures are preserved by modifying the model architecture itself with the use of residual connections. The utility of PPNN [36] has previously been demonstrated in terms of training complexity and extrapolability for a few spatiotemporal dynamic problems.…”