Multi-Facet Clustering Variational Autoencoders

Abstract

Work in deep clustering focuses on finding a single partition of data. However, high-dimensional data, such as images, typically feature multiple interesting characteristics one could cluster over. For example, images of objects against a background could be clustered over the shape of the object and separately by the colour of the background. In this paper, we introduce Multi-Facet Clustering Variational Autoencoders (MFCVAE), a novel class of variational autoencoders with a hierarchy of latent variables, each with a Mixture-of-Gaussians prior, that learns multiple clusterings simultaneously, and is trained fully unsupervised and end-to-end. MFCVAE uses a progressivelytrained ladder architecture which leads to highly stable performance. We provide novel theoretical results for optimising the ELBO analytically with respect to the categorical variational posterior distribution, and corrects earlier influential theoretical work. On image benchmarks, we demonstrate that our approach separates out and clusters over different aspects of the data in a disentangled manner. We also show other advantages of our model: the compositionality of its latent space and that it provides controlled generation of samples.

Integration of scRNA-seq data by disentangled representation learning with condition domain adaptation

“…( 11 ), the distribution follows a Gaussian distribution . Assuming the separability of the biological latent space and the condition-specific biological latent space, our Multi-facet variational distribution [ 30 ] adopts a mean-field form: where J =2. For each j , the variational distribution takes the form of a multivariate Gaussian distribution featuring a diagonal covariance matrix, and its mean and variance are determined by the encoder , , and .…”

Multidimensional clustering with Bayesian networks

“…Second, we can speed up GLSL by using parallelization (e.g., by simultaneously evaluating multiple latent operators). Finally, we can investigate a variant of variational autoencoders where there is a latent superstructure with multiple categorical latent variables Falck et al, 2021] that is learned using GLSL. The winner in each row is highlighted in blue.…”

“…There is a growing interest in combining complex LVMs with unsupervised deep learning models Falck et al, 2021]. In this context, we can investigate a variant of variational autoencoders whose latent superstructure is learned using our method from Chapter 6.…”