Machine learning structure preserving brackets for forecasting irreversible processes

Abstract

Forecasting of time-series data requires imposition of inductive biases to obtain predictive extrapolation, and recent works have imposed Hamiltonian/Lagrangian form to preserve structure for systems with reversible dynamics. In this work we present a novel parameterization of dissipative brackets from metriplectic dynamical systems appropriate for learning irreversible dynamics with unknown a priori model form. The process learns generalized Casimirs for energy and entropy guaranteed to be conserved and nondecreasing, respectively. Furthermore, for the case of added thermal noise, we guarantee exact preservation of a fluctuation-dissipation theorem, ensuring thermodynamic consistency. We provide benchmarks for dissipative systems demonstrating learned dynamics are more robust and generalize better than either "black-box" or penalty-based approaches. Background and previous workModeling time-series data as a solution to a dynamical system with learnable dynamics has been shown to be effective in both data-driven modeling for physical systems and traditional machine learning (ML) tasks. Broadly, it has been observed that imposition of physics-based structure leads to more robust architectures which generalize well [1]. On one end of the spectrum of inductive biases, universal differential equations (UDE) [2] assume an a priori known model form, thus imposing the strongest bias. On the other, neural ordinary differential equations (NODEs) [3] assume a completely black-box model form with minimal bias.Many recent approaches have turned to structure preserving models of reversible dynamics to obtain an inductive bias that lies in between [4,5,6,7,8]. One may use black-box deep neural networks (DNNs) to learn an energy of a system with unknown model form, while the algebraic structure of Hamiltonian/Lagrangian dynamics provides a flow map which conserves energy. Typically, the learned flow map has symplectic structure so that phase space trajectories are conserved. In classification problems, this mitigates the vanishing/exploding gradient problem and improves accuracy [9]; in physics, this guarantees that extrapolated states are physically realizable [10].Such approaches are only appropriate for reversible systems lacking friction or dissipation. In the physics literature, the theory of metriplectic dynamical systems provides a generalization of the Poisson brackets of Hamiltonian/Lagrangian mechanics which model not just a conserved energy, but generalized Casimirs such as entropy [11,12]. Physical systems which can be cast in this framework Preprint. Under review.

Structure-preserving Sparse Identification of Nonlinear Dynamics for Data-driven Modeling

“…Parameterization for GENERIC Here, we review the parameterization technique, developed in (Oettinger 2014) and further extended into deep learning settings in (Lee, Trask, and Stinis 2021). As for the Hamiltonian Eq.…”

“…As noted in (Lee, Trask, and Stinis 2021), with canonical coordinates x = [q, p] T , and canonical Poisson matrix L = , and M = 0, the GENERIC formalism in Eq. ( 4) recovers Hamiltonian dynamics.…”

“…image/language processing tasks, engineering and science models pose strict requirements on physical quantities to guarantee properties such as conservation, thermodynamic consistency and well-posedness of resulting models (Baker et al 2019). These structural constraints translate to desirable mathematical properties for simulation, such as improved numerical stability and accuracy, particularly for chaotic systems (Lee, Trask, and Stinis 2021;Trask, Huang, and Hu 2020).…”

“…While so-called physics-informed ML (PIML) approaches seek to impose these properties by imposing soft physics constraints into the ML process, many applications require structure preservation to hold exactly; PIML requires empirical tuning of weighting parameters and physics properties hold only to within optimization error, which typically may be large (Wang, Teng, and Perdikaris 2020;Rohrhofer, Posch, and Geiger 2021). Structure-preserving machine learning has emerged as a means of designing architectures such that physics constraints hold exactly by construction (Lee, Trask, and Stinis 2021;Trask, Huang, and Hu 2020). By parameterizing relevant geometric or topological structures, researchers obtain more data-efficient hybrid physics/ML architectures with guaranteed mathematical properties.…”

“…Reversible bracket formalisms (e.g. Hamiltonian/Lagrangian mechanics) have been shown effective for learning reversible dynamics (Toth et al 2019;Cranmer et al 2020;Lutter, Ritter, and Peters 2018;Chen et al 2019;Jin et al 2020;Tong et al 2021;Zhong, Dey, and Chakraborty 2021;Chen and Tao 2021;Bertalan et al 2019), while dissipative metric bracket extensions provide generalizations to irreversible dynamics in the metriplectic formalism (Lee, Trask, and Stinis 2021;Desai et al 2021;Hernández et al 2021;Yu et al 2020;Zhong, Dey, and Chakraborty 2020). Accounting for dissipation in this manner provides important thermodynamic consistency guarantees: namely, discrete versions of the first and second law of thermodynamics along with a fluctuation-dissipation theorem for thermal systems at microscales.…”