Simplifying Hamiltonian and Lagrangian Neural Networks via Explicit Constraints

Finzi, Marc; Wang, Ke Alexander; Wilson, Andrew Gordon

doi:10.48550/arxiv.2010.13581

Cited by 7 publications

(8 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Using equation (19), this implies that for any δ > 0, with probability at least 1 − δ, the following holds for all θ ∈ Θ:…”

Section: A12 Putting Results Togethermentioning

confidence: 99%

“…Most relevant to the types of problems we focus on is roto-translational equivariance [50,54,20,39], applications of GNNs in physical settings [9,5,26,2,1,37,34,38,13] and the encoding of time-invariance in RNNs [17,24,23,10,48]. Recent works have encoded Hamiltonian and Lagrangian mechanics into neural models [22,30,11,19], with gains in data-efficiency in physical and robotics systems, including some modeling controlled or dissipative systems [60,15]. In contrast to these works, we propose to encode biases via predictiontime fine-tuning following the tailoring framework [3], instead of architectural constraints.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Noether Networks: Meta-Learning Useful Conserved Quantities

Alet¹,

Doblar²,

Zhou³

et al. 2021

Preprint

View full text Add to dashboard Cite

Progress in machine learning (ML) stems from a combination of data availability, computational resources, and an appropriate encoding of inductive biases. Useful biases often exploit symmetries in the prediction problem, such as convolutional networks relying on translation equivariance. Automatically discovering these useful symmetries holds the potential to greatly improve the performance of ML systems, but still remains a challenge. In this work, we focus on sequential prediction problems and take inspiration from Noether's theorem to reduce the problem of finding inductive biases to meta-learning useful conserved quantities. We propose Noether Networks: a new type of architecture where a meta-learned conservation loss is optimized inside the prediction function. We show, theoretically and experimentally, that Noether Networks improve prediction quality, providing a general framework for discovering inductive biases in sequential problems.

show abstract

“…Using equation (19), this implies that for any δ > 0, with probability at least 1 − δ, the following holds for all θ ∈ Θ:…”

Section: A12 Putting Results Togethermentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Noether Networks: Meta-Learning Useful Conserved Quantities

Alet¹,

Doblar²,

Zhou³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In contrast to policy learning methods that seek to adapt to environment dynamics, a parallel line of work has explored directly modeling these system dynamics as learnable ordinary differential equations (ODEs) with deep neural networks. While the majority of these works seek to model arbitrary physical dynamics [22,23,39,40], some have explored specific complex physical phenomena such as contact [41][42][43] and fluid dynamics [44,45]. These works leverage strong physical priors to enforce model plausibility.…”

Section: Related Workmentioning

confidence: 99%

OSCAR: Data-Driven Operational Space Control for Adaptive and Robust Robot Manipulation

Wong¹,

Makoviychuk²,

Anandkumar³

et al. 2021

Preprint

View full text Add to dashboard Cite

Learning performant robot manipulation policies can be challenging due to high-dimensional continuous actions and complex physics-based dynamics. This can be alleviated through intelligent choice of action space. Operational Space Control (OSC) has been used as an effective taskspace controller for manipulation. Nonetheless, its strength depends on the underlying modeling fidelity, and is prone to failure when there are modeling errors. In this work, we propose OSC for Adaptation and Robustness (OSCAR), a data-driven variant of OSC that compensates for modeling errors by inferring relevant dynamics parameters from online trajectories. OSCAR decomposes dynamics learning into task-agnostic and task-specific phases, decoupling the dynamics dependencies of the robot and the extrinsics due to its environment. This structure enables robust zero-shot performance under out-of-distribution and rapid adaptation to significant domain shifts through additional finetuning. We evaluate our method on a variety of simulated manipulation problems, and find substantial improvements over an array of controller baselines. For more results and information, please visit https://cremebrule.github.io/oscar-web/.

show abstract

“…Combining learning algorithms with principles from physics and numerical methods, such as auxiliary loss terms and rich inductive biases, can improve sample complexity, computational efficiency, and generalization (Wu et al, 2018;Karniadakis et al, 2021;Chen et al, 2018;Rubanova et al, 2019). Imposing Hamiltonian (Greydanus et al, 2019;Sanchez-Gonzalez et al, 2019; and Lagrangian (Lutter et al, 2019;Cranmer et al, 2020;Finzi et al, 2020) mechanics in learned simulators offers unique speed/accuracy tradeoffs and can preserve symmetries more effectively.…”

Section: Background and Related Workmentioning

confidence: 99%

Constraint-based graph network simulator

Rubanova¹,

Sánchez‐González²,

Pfaff³

et al. 2021

Preprint

View full text Add to dashboard Cite

In the rapidly advancing area of learned physical simulators, nearly all methods train forward models that directly predict future states from input states. However, many traditional simulation engines use a constraint-based approach instead of direct prediction. Here we present a framework for constraint-based learned simulation, where a scalar constraint function is implemented as a neural network, and future predictions are computed as the solutions to optimization problems under these learned constraints. We implement our method using a graph neural network as the constraint function and gradient descent as the constraint solver. The architecture can be trained by standard backpropagation. We test the model on a variety of challenging physical domains, including simulated ropes, bouncing balls, colliding irregular shapes and splashing fluids. Our model achieves better or comparable performance to top learned simulators. A key advantage of our model is the ability to generalize to more solver iterations at test time to improve the simulation accuracy. We also show how hand-designed constraints can be added at test time to satisfy objectives which were not present in the training data, which is not possible with forward approaches. Our constraint-based framework is applicable to any setting where forward learned simulators are used, and demonstrates how learned simulators can leverage additional inductive biases as well as the techniques from the field of numerical methods. INTRODUCTIONConsider a bowling ball colliding with a bowling pin. You might explain this event as involving a pair of forces being generated, one which causes the pin to move, and the other which causes the ball to careen away with a different direction and speed. This kind of intuitive cause-and-effect approach is analogous to physical simulators that apply an explicit forward model to calculate a future state directly from the current one, such as when numerically integrating discretized equations of motion.An alternative, but equally valid, way to explain the collision is in terms of constraint satisfaction: the ball and pin cannot occupy the same location at the same time, and their combined energies and momenta must be conserved. The post-collision trajectories are the only way the future can unfold without violating these constraints. This constraint-based approach is analogous to physical simulators that use an implicit function to model a system of constraints over the current and future states. These simulators generate a prediction by searching for a future state that respects all constraints.Both families of simulators-those based on explicit, forward functions versus those which define the dynamics implicitly, via constraints-are widely used in physics, engineering, and graphics. In principle they can model the same types of dynamics, however, they differ in the ways they compute their respective predictions. In practice these simulators strike different trade-offs that determine why one or the other is preferred in different dom...

show abstract

Simplifying Hamiltonian and Lagrangian Neural Networks via Explicit Constraints

Cited by 7 publications

References 12 publications

Noether Networks: Meta-Learning Useful Conserved Quantities

Noether Networks: Meta-Learning Useful Conserved Quantities

OSCAR: Data-Driven Operational Space Control for Adaptive and Robust Robot Manipulation

Constraint-based graph network simulator

Contact Info

Product

Resources

About