Stacked Capsule Autoencoders

Abstract

An object can be seen as a geometrically organized set of interrelated parts. A system that makes explicit use of these geometric relationships to recognize objects should be naturally robust to changes in viewpoint, because the intrinsic geometric relationships are viewpoint-invariant. We describe an unsupervised version of capsule networks, in which a neural encoder, which looks at all of the parts, is used to infer the presence and poses of object capsules. The encoder is trained by backpropagating through a decoder, which predicts the pose of each already discovered part using a mixture of pose predictions. The parts are discovered directly from an image, in a similar manner, by using a neural encoder, which infers parts and their affine transformations. The corresponding decoder models each image pixel as a mixture of predictions made by affine-transformed parts. We learn object-and their part-capsules on unlabeled data, and then cluster the vectors of presences of object capsules. When told the names of these clusters, we achieve state-of-the-art results for unsupervised classification on SVHN (55%) and near state-of-the-art on MNIST (98.5%). * This work was done during an internship at Google Brain. 2 This may explain why accessing perceptual knowledge about objects, when they are not visible, requires creating a mental image of the object with a specific viewpoint.Preprint. Under review.

Unsupervised learning of features and object boundaries from local prediction

“…To improve biological plausibility, all computations in our model are local and all units are connected to the same small, local set of other units throughout learning and inference, which matches early visual cortex, in which the lateral connections that follow natural image statistics are implemented anatomically [4,26,48,59]. This in contrast to other ideas that require flexible pointers to arbitrary locations and features [as discussed by 53] or capsules that flexibly encode different parts of the input [9,36,50,51]. Nonetheless, we employ contrastive learning objectives and backpropagation here, for which we do not provide a biologically plausible implementations.…”

Holographic-(V)AE: an end-to-end SO(3)-Equivariant (Variational) Autoencoder in Fourier Space

“…A small body of work focuses on developing equivariant autoencoders. Several methods construct data and group-specific architectures to auto-encode data equivariantly, learning an equivariant representation in the process (Hinton et al, 2011;Kosiorek et al, 2019). Others use supervision to extract class-invariant and class-equivariant representations (Feige, 2022).…”

Graph Capsule Aggregation for Unaligned Multimodal Sequences

“…Capsule Network is first proposed in [22] and is improved in [7] and [12], which is designed for image features extraction. In general, Capsule Network can not only effectively fuse information from numerous elements into highly expressive representations without information loss, but also reveal the contributions from different elements to the representations by routing mechanism.…”