Approximately Equivariant Networks for Imperfectly Symmetric Dynamics

Abstract

Incorporating symmetry as an inductive bias into neural network architecture has led to improvements in generalization, data efficiency, and physical consistency in dynamics modeling. Methods such as CNNs or equivariant neural networks use weight tying to enforce symmetries such as shift invariance or rotational equivariance. However, despite the fact that physical laws obey many symmetries, real-world dynamical data rarely conforms to strict mathematical symmetry either due to noisy or incomplete data or to symmetry breaking features in the underlying dynamical system. We explore approximately equivariant networks which are biased towards preserving symmetry but are not strictly constrained to do so. By relaxing equivariance constraints, we find that our models can outperform both baselines with no symmetry bias and baselines with overly strict symmetry in both simulated turbulence domains and real-world multi-stream jet flow.

Relaxing Equivariance Constraints with Non-stationary Continuous Filters

“…This is effective for sparsely sampled subgroups, such as rotation, but becomes less practical for more densely sampled groups such as translation where the local support could lead to very sparse feature maps. Lastly, Wang et al [28] propose a relaxed convolutional operator similar to our non-stationary approach, but require a low-rank factorization which potentially limits the expressivity of the feature maps. The same work briefly discusses a mathematical description without such factorization similar to our proposal which is deemed "too large a trainable weight space to be practical".…”

“…Symmetries play a central role in physics. In [28] approximate equivariances are used to allow more robust models for dynamical systems. In Noether networks [1] conservation laws are inferred by learning symmetries from data.…”

Data Augmentation vs. Equivariant Networks: A Theory of Generalization on Dynamics Forecasting

“…Thus, some work has recently explored the idea of building approximately equivariant models and empirically demonstrated the benefits of it in modeling real-world data (van der Ouderaa et al, 2022;Romero & Lohit, 2021;Finzi et al, 2021). For example, (Wang et al, 2022) designed approximately equivariant models by relaxing the weight sharing schemes in the equivariant convolution networks.…”

“…We show that when the underlying dynamics is symmetric, equivariant networks achieve a tighter generalization bound than data augmentation. Furthermore, when the symmetries in the data are only approximate, the generalization bound for approximately equivariant networks (Wang et al, 2022) is further improved.…”

“…Several recent work have designed approximately equivariant networks (Wang et al, 2022;van der Ouderaa et al, 2022;Finzi et al, 2021) to learn the approximate functions.…”