Efficient Generalized Spherical CNNs

Abstract

Many problems across computer vision and the natural sciences require the analysis of spherical data, for which representations may be learned efficiently by encoding equivariance to rotational symmetries. We present a generalized spherical CNN framework that encompasses various existing approaches and allows them to be leveraged alongside each other. The only existing non-linear spherical CNN layer that is strictly equivariant has complexity OpC 2 L 5 q, where C is a measure of representational capacity and L the spherical harmonic bandlimit. Such a high computational cost often prohibits the use of strictly equivariant spherical CNNs. We develop two new strictly equivariant layers with reduced complexity OpCL 4 q and OpCL 3 log Lq, making larger, more expressive models computationally feasible. Moreover, we adopt efficient sampling theory to achieve further computational savings. We show that these developments allow the construction of more expressive hybrid models that achieve state-of-the-art accuracy and parameter efficiency on spherical benchmark problems.

Holographic-(V)AE: an end-to-end SO(3)-Equivariant (Variational) Autoencoder in Fourier Space

“…A.4.3) and evaluate them similarly to the MNIST dataset and compute the Shrec17 retrieval metrics via the latent space linear classifier’s predictions (Table 1). H-AE achieves the best classification and retrieval results for autoencoder-based models (Lohit & Trivedi, 2020), and is competitive with supervised models (Esteves et al, 2020; Cobb et al, 2021) despite the lower grid bandwidth and the small latent space. Using KNN classification instead of a linear classifier further improves performance (Table A.2).…”

“…Specifically, pairs of features with any degree pair (𝓁 1 , 𝓁 2 ) may be used to produce a feature of degree 𝓁 3 as long as | 𝓁 1 𝓁 2 | ≤ 𝓁 3 ≤ 𝓁 1 + 𝓁 2 . Features of the same degree are then concatenated to produce the final equivariant (steerable) output tensor. Since each produced feature (often referred to as a “fragment” in the literature Kondor et al (2018); Cobb et al (2021)) is independently equivariant, computing only a subset of them still results in an equivariant output, albeit with lower representational power. Reducing the number of computed fragments is desirable since their computation cannot be easily parallelized.…”

“…In other words, to reduce complexity we should identify a small subset of fragments that can still offer sufficient representational power. In this work we adopt the “MST subset” solution proposed by Cobb et al (2021), which adopts the following strategy: when computing features of the same degree 𝓁 3 , exclude the degree pair (𝓁 0 , 𝓁 2 ) if the (𝓁 0 , 𝓁 1 ) and the (𝓁 1 , 𝓁 2 ) pairs have already been computed. The underlying assumption behind this solution is that the last two pairs already contain some information about the first pair, thus making its computation redundant. The resulting subset of pairs can be efficiently computed via the Minimum Spanning Tree of the graph describing the possible pairs used to generate features of a single degree 𝓁 3 , given the maximum desired degree 𝓁 max .…”

Geometric Deep Learning and Equivariant Neural Networks

“…They yield a feature map transforming in the tensor product representation which is then decomposed into irreducible representations. To eschew the large tensors in this process, [24,30] introduce various refinements of this basic idea.…”

“…The Clebsch-Gordan nets introduced in [23] have a similar structure but use as nonlinearities tensor products in the Fourier domain, instead of point-wise nonlinearities in the spatial domain. Several modifications of this approach led to a more efficient implementation in [24]. The constructions mentioned so far involve convolutions which map spherical features to features defined on SO(3).…”