Structure preserving deep learning

Celledoni, Elena; Ehrhardt, Matthias J.; Etmann, Christian; McLachlan, Robert I; Owren, Brynjulf; Schönlieb, Carola‐Bibiane; Sherry, Ferdia

doi:10.48550/arxiv.2006.03364

Cited by 6 publications

(7 citation statements)

References 64 publications

(122 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus, the learned time integrator should fulfill the first principles of physics to provide credible predictions of future events to help in decision-making. By structure-preserving neural networks (SPNN) we refer to a class of techniques that are constructed to satisfy some a priori known properties of the problem such as equivariance [71] or energy conservation [57] [60]. In their most general form, they can be applied to conservative as well as dissipative problems, in which the principles of thermodynamics are satisfied by construction [39] [38].…”

Section: Learning the Dynamical Evolution Based On Structure-preservi...mentioning

confidence: 99%

Physics Perception in Sloshing Scenes With Guaranteed Thermodynamic Consistency

Moya

Badías

González

et al. 2023

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Physics perception very often faces the problem that only limited data or partial measurements on the scene are available. In this work, we propose a strategy to learn the full state of sloshing liquids from measurements of the free surface. Our approach is based on recurrent neural networks (RNN) that project the limited information available to a reduced-order manifold to not only reconstruct the unknown information but also be capable of performing fluid reasoning about future scenarios in real-time. To obtain physically consistent predictions, we train deep neural networks on the reduced-order manifold that, through the employ of inductive biases, ensure the fulfillment of the principles of thermodynamics. RNNs learn from history the required hidden information to correlate the limited information with the latent space where the simulation occurs. Finally, a decoder returns data to the high-dimensional manifold, to provide the user with insightful information in the form of augmented reality. This algorithm is connected to a computer vision system to test the performance of the proposed methodology with real information, resulting in a system capable of understanding and predicting future states of the observed fluid in real-time.

show abstract

Section: Learning the Dynamical Evolution Based On Structure-preservi...mentioning

confidence: 99%

Physics Perception in Sloshing Scenes With Guaranteed Thermodynamic Consistency

Moya

Badías

González

et al. 2023

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

show abstract

“…Note that we do not use the standard Tikhonov regularization (so called weight decay) on the weights as they do not guarantee smoothness in time which is crucial for reversible networks and integration in time (Celledoni et al, 2020).…”

Section: Regularizationmentioning

confidence: 99%

Mimetic Neural Networks: A unified framework for Protein Design and Folding

Eliasof¹,

Boesen²,

Haber³

et al. 2021

Preprint

View full text Add to dashboard Cite

Recent advancements in machine learning techniques for protein folding motivate better results in its inverse problem -protein design. In this work we introduce a new graph mimetic neural network, MimNet, and show that it is possible to build a reversible architecture that solves the structure and design problems in tandem, allowing to improve protein design when the structure is better estimated. We use the ProteinNet data set and show that the state of the art results in protein design can be improved, given recent architectures for protein folding.

show abstract

“…Thus, it is fitting that a geometric integrator is used to solve optimal control problem to guarantee that the numerical solution is representative of the control system. First, we describe how optimal is used to formulate the training of deep neural network in section 2.3 and, here, we adhere to the approach of [10,20,67]. The resultant Hamilton's equations are then solved numerically using structure-preserving integrator.…”

Section: Hamilton-jacobi Integrators For Deep Learningmentioning

confidence: 99%

Deep Learning: Hydrodynamics, and Lie-Poisson Hamilton-Jacobi Theory

Ganaba

2021

Preprint

View full text Add to dashboard Cite

The interpretation of deep learning as a dynamical system has gained a considerable attention in recent years as it provides a promising framework. It allows for the use of existing ideas from established fields of mathematics for studying deep neural networks. In this article we present deep learning as an equivalent hydrodynamics formulation which in fact possess a Lie-Poisson structure and this further allows for using Poisson geometry to study deep learning. This is possible by considering the training of a deep neural network as a stochastic optimal control problem, which is solved using mean-field type control. The optimality conditions for the stochastic optimal control problem yield a system of partial differential equations, which we reduce to a system of equations of quantum hydrodynamics. We further take the hydrodynamics equivalence to show that, with conditions imposed on the network, that it is equivalent to the nonlinear Schrödinger equation.As the such systems have a particular geometric structure, we also present a structure-preserving numerical scheme based on the Hamilton-Jacobi theory and its Lie-Poisson version known as Lie-Poisson Hamilton-Jacobi theory. The choice of this scheme was decided by the fact it can be applied to both deterministic and mean-field type control Hamiltonians. Not only this method eliminates the possibility of the emergence of fictitious solutions, it also eliminates the need for additional constraints when constructing high order methods.

show abstract

Structure preserving deep learning

Cited by 6 publications

References 64 publications

Physics Perception in Sloshing Scenes With Guaranteed Thermodynamic Consistency

Physics Perception in Sloshing Scenes With Guaranteed Thermodynamic Consistency

Mimetic Neural Networks: A unified framework for Protein Design and Folding

Deep Learning: Hydrodynamics, and Lie-Poisson Hamilton-Jacobi Theory

Contact Info

Product

Resources

About