Unsupervised Disentangled Representation Learning with Analogical Relations

Abstract

Learning the disentangled representation of interpretable generative factors of data is one of the foundations to allow artificial intelligence to think like people. In this paper, we propose the analogical training strategy for the unsupervised disentangled representation learning in generative models. The analogy is one of the typical cognitive processes, and our proposed strategy is based on the observation that sample pairs in which one is different from the other in one specific generative factor show the same analogical relation. Thus, the generator is trained to generate sample pairs from which a designed classifier can identify the underlying analogical relation. In addition, we propose a disentanglement metric called the subspace score, which is inspired by subspace learning methods and does not require supervised information. Experiments show that our proposed training strategy allows the generative models to find the disentangled factors, and that our methods can give competitive performances as compared with the state-of-the-art methods.

Learning Disentangled Representation with Pairwise Independence

“…Particularly, we compare how well the methods capture the attributes in CelebA dataset by examining the maximum correlations and the prediction performances in canonical correlation analysis. We also compare the methods along subspace score (Li, Tang, and He 2018), an unsupervised disentanglement metric. Furthermore, we display rerendered sample sequences in the latent traversal as appropriate.…”

“…These are methods based on the mutual independence assumption. We also include comparisons with AnaVAE (Li, Tang, and He 2018) and InfoGAN (Chen et al 2016). VAE (Kingma and Welling 2014) is also compared as a baseline.…”

“…in (2) The assumption behind VAE and these variants is that interpretable factors are mutually independent and can be captured with the factorial distribution p(z). Other works (Chen et al 2016;Li, Tang, and He 2018) encourage disentanglement by minimizing the mutual information between x and z with extra components of the model.…”

“…In this part we present the comparison along subspace score (Li, Tang, and He 2018). Subspace score is an unsupervised disentanglement metric.…”

“…Kim and Mnih (2018) and Chen et al (2018) optimize the term of total correlation to enable the distribution to be factorial. Other works (Chen et al 2016;Li, Tang, and He 2018) take the principle of mutual information minimization. Most of these methods are built on top of variational autoencoder (Kingma and Welling 2014).…”