Low Dimensional Invariant Embeddings for Universal Geometric Learning

Abstract

This paper studies separating invariants: mappings on d-dimensional semi-algebraic subsets of D dimensional Euclidean domains which are invariant to semi-algebraic group actions and separate orbits. The motivation for this study comes from the usefulness of separating invariants in proving universality of equivariant neural network architectures.We observe that in several cases the cardinality of separating invariants proposed in the machine learning literature is much larger than the ambient dimension D. As a result, the theoretical universal constructions based on these separating invariants is unrealistically large. Our goal in this paper is to resolve this issue.We show that when a continuous family of semi-algebraic separating invariants is available, separation can be obtained by randomly selecting 2d+1 of these invariants. We apply this methodology to obtain an efficient scheme for computing separating invariants for several classical group actions which have been studied in the invariant learning literature. Examples include matrix multiplication actions on point clouds by permutations, rotations, and various other linear groups.

Group-invariant max filtering

“…(Of course, any single nonzero template suffices in this case.) Theorem 12 is an improvement to Theorem 1.9 in [37], which gives the condition n ≥ 2d + 1. We obtain the improvement n ≥ 2d/k by leveraging a more detailed notion of strongly separating, as well as the positive homogeneity of max filtering.…”

“…and so dim(π −1 2 (x, y)) ≤ n(d − k), since the max filtering map is k-strongly separating by assumption. We use the fact that π 2 (L) is contained in the unit sphere in (R d ) 2 together with Lemma 1.10 in [37] to obtain…”

“…This work was initiated at the SOFT 2021: Summer of Frame Theory virtual workshop. The authors thank Boris Alexeev for helpful discussions and Soledad Villar for bringing the article [37] to their attention.…”

“…In Section 2, we define max filters, we identify some basic properties, and we highlight a few familiar examples. In Section 3, we use ideas from [37] to establish that 2d generic max filters separate all G-orbits when G is finite (see Corollary 13), and then we describe various settings in which max filtering is computationally efficient. In Section 4, we show that when G is finite, a sufficiently large random max filter bank is bilipschitz with high probability; see Theorem 18.…”

Algebraic Theory of Phase Retrieval

“…, w N are the rows of the matrix representing A with respect to our chosen basis for R N . In particular, if x ∈ R N then, using the notation of [22], b Ax = P (x; A), where…”