Physics-Informed Probabilistic Learning of Linear Embeddings of Nonlinear Dynamics with Guaranteed Stability

Abstract

The Koopman operator has emerged as a powerful tool for the analysis of nonlinear dynamical systems as it provides coordinate transformations which can globally linearize the dynamics. Recent deep learning approaches such as Linearly-Recurrent Autoencoder Networks (LRAN) show great promise for discovering the Koopman operator for a general nonlinear dynamical system from a data-driven perspective, but several challenges remain. In this work, we formalize the problem of learning the continuous-time Koopman operator with deep neural networks in a measure-theoretic framework. This induces two types of models: differential and recurrent form, the choice of which depends on the availability of the governing equations and data. We then enforce a structural parameterization that renders the realization of the Koopman operator provably stable. A new autoencoder architecture is constructed, such that only the residual of the dynamic mode decomposition is learned. Finally, we employ mean-field variational inference (MFVI) on the aforementioned framework in a hierarchical Bayesian setting to quantify uncertainties in the characterization and prediction of the dynamics of observables. The framework is evaluated on a simple polynomial system, the Duffing oscillator, and an unstable cylinder wake flow with noisy measurements.

Toward stable, general machine-learned models of the atmospheric chemical system

“…In effect, we create a model that is more "stable" and "general" than previous results, two key prerequisites for routine use in atmospheric modeling. We do this by leveraging three key innovations: 1) the use of a recurrent training regime (Brenowitz & Bretherton, 2018;McGibbon & Bretherton, 2019) to train the model to represent chemical reactions across multiple time scales, 2) the use of an encoder-operator-decoder framework to reduce the dimensionality of the chemical system (Lee & Carlberg, 2018;Pan & Duraisamy, 2019;Regazzoni et al, 2019), and 3) the use of a weighted optimization metric to create surrogate models that specialize in the prediction of key air quality metrics such as ozone and total particulate matter (PM).…”

Sparsity-promoting algorithms for the discovery of informative Koopman-invariant subspaces

“…As shown in table 2, nearly half of the accurate KDMD eigenmodes identified are removed with the proposed sparse feature selection. Note that for all three cases, the number of selected modes (around 32 to 34) is still larger than that required in neural network models (around 10) (Otto & Rowley 2019;Pan & Duraisamy 2020). This is because the subspace spanned by KDMD/EDMD relies on a predetermined dictionary rather than being data-adaptive like neural network models.…”

“…Although the use of a kernel defines an infinite-dimensional feature space, the resulting finite number of effective features can still be affected by both the type of kernel and the hyperparameters in the kernel, as clearly shown by Kutz et al (2016). Compared with EDMD/KDMD, which are based on a fixed dictionary of features, neural network approaches (Lusch et al 2018;Otto & Rowley 2019;Pan & Duraisamy 2020) have the potential to be more expressive in searching for a larger Koopman-invariant subspace. From a kernel viewpoint (Cho & Saul 2009), feedforward neural networks enable adaptation of the kernel function to the data.…”

Data-driven recovery of hidden physics in reduced order modeling of fluid flows

“…However, the machine learning methods and in particular deep learning models lack interpretability and are prone to produce physically inconsistent results. Hence, there is an active research going on to incorporate the physical models in machine learning methods to make them physically consistent, such as loss regularization based on physical laws [14], designing novel neural network to embed certain physical properties [15], and building hybrid models to correct the imperfect knowledge in physical models [16,17]. How should we inject physics and domain knowledge into machine learning models?…”