Tagger: Deep Unsupervised Perceptual Grouping

Abstract

We present a framework for efficient perceptual inference that explicitly reasons about the segmentation of its inputs and features. Rather than being trained for any specific segmentation, our framework learns the grouping process in an unsupervised manner or alongside any supervised task. We enable a neural network to group the representations of different objects in an iterative manner through a differentiable mechanism. We achieve very fast convergence by allowing the system to amortize the joint iterative inference of the groupings and their representations. In contrast to many other recently proposed methods for addressing multi-object scenes, our system does not assume the inputs to be images and can therefore directly handle other modalities. We evaluate our method on multi-digit classification of very cluttered images that require texture segmentation. Remarkably our method achieves improved classification performance over convolutional networks despite being fully connected, by making use of the grouping mechanism. Furthermore, we observe that our system greatly improves upon the semi-supervised result of a baseline Ladder network on our dataset. These results are evidence that grouping is a powerful tool that can help to improve sample efficiency.

Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders

“…This has been recently achieved with InfoGANs (Chen et al, 2016a), where structured latent variables are included as part of the noise vector, and the mutual information between these latent variables and the generator distribution is then maximised as a mini-max game between the two networks. Similarly, Tagger (Greff et al, 2016), which combines iterative amortized grouping and ladder networks, aims to perceptually group objects in images by iteratively denoising its inputs and assigning parts of the reconstruction to different groups. introduced a way to combine amortized inference with stochastic variational inference in an algorithm called structured VAEs.…”

Towards Deep Symbolic Reinforcement Learning

“…Deep neural networks in particular have proven to be remarkably effective for supervised learning from large datasets using backpropagation [1,2]. Deep learning is therefore already a viable solution to the symbol grounding problem in the supervised case, and for the unsupervised case, which is essential for a full solution, rapid progress is being made [20,21,22,23,24]. The hybrid neuralsymbolic reinforcement learning architecture we propose relies on a deep learning solution to the symbol grounding problem.…”

Unsupervised Discovery of Object Radiance Fields

“…Our method is also closely related to recent work on deep probabilistic inference for scene decomposition. Most works formulate the problem as compositional generative models, in which a visual scene is represented by a set of latent codes that either correspond to localized object-centric patches [11,8,24,28,19] or scene mixture components [2,15,16,17,10]. The scene mixture models generate full-sized images for each latent code and blend them via attentional masks [2] in iterative variational inference frameworks.…”