Steerable Partial Differential Operators for Equivariant Neural Networks

Abstract

Recent work in equivariant deep learning bears strong similarities to physics. Fields over a base space are fundamental entities in both subjects, as are equivariant maps between these fields. In deep learning, however, these maps are usually defined by convolutions with a kernel, whereas they are partial differential operators (PDOs) in physics. Developing the theory of equivariant PDOs in the context of deep learning could bring these subjects even closer together and lead to a stronger flow of ideas. In this work, we derive a G-steerability constraint that completely characterizes when a PDO between feature vector fields is equivariant, for arbitrary symmetry groups G. We then fully solve this constraint for several important groups. We use our solutions as equivariant drop-in replacements for convolutional layers and benchmark them in that role. Finally, we develop a framework for equivariant maps based on Schwartz distributions that unifies classical convolutions and differential operators and gives insight about the relation between the two. However, one remaining difference is that physics uses equivariant partial differential operators (PDOs) to define maps between fields, such as the gradient or Laplacian. Therefore, using PDOs instead of convolutions in deep learning would complete the analogy to physics and could lead to even more transfer of ideas between subjects.Equivariant PDO-based networks have already been designed in prior work [21][22][23]. Most relevant for our work are PDO-eConvs [22], which can be seen as the PDO-analogon of group convolutions. * Work done during an internship at QUVA Lab Preprint. Under review.

PDE-Based Group Equivariant Convolutional Neural Networks

“…More recent work in this direction includes integrating equivariant partial differential operators in steerable CNNs [75], drawing a strong analogy between deep learning and physics.…”

Leveraging symmetries in pick and place

“…That is, if F takes an image f rotated by g (RHS of equation ( 2)), then it is possible to recover the same output by evaluating F for the un-rotated image f and rotating its output (LHS of equation ( 2)). The most equivariant mappings between spaces of feature fields are convolutions with G-steerable kernels (Weiler et al, 2018;Jenner and Weiler, 2021). Denote the input field type as ρ in : G → R d in ×d in and the output field type as ρ out : G → R d out ×d out .…”

PDO-s3DCNNs: Partial Differential Operator Based Steerable 3D CNNs

“…Gaussian Discretization We can also employ the derivatives of Gaussian function for estimation (Jenner & Weiler, 2021): given points…”

“…Besides, as proven in (Finzi et al, 2021), scale-like nonlinearities are not sufficient for universality, and can limit the model performance. These disadvantages also account for why most related works (Weiler & Cesa, 2019;Jenner & Weiler, 2021) employ regular or quotient representations of discrete subgroups for implementation…”

“…We emphasize that our work is not a trivial extension of PDO-eConv (Shen et al, 2020) to the 3D case, because PDO-eConv is limited to the regular features of discrete subgroups, and cannot address other feature fields and continuous groups whereas ours can. Jenner & Weiler (2021) successfully proposed steerabble PDOs in the 2D case. However, their theory cannot deal with arbitrary 3D rotation groups and their representations, such as discrete subgroups, due to the complexity of 3D rotations.…”