Disentangling by Factorising

Kim, Hyunjik; Mnih, Andriy

doi:10.48550/arxiv.1802.05983

Cited by 131 publications

(247 citation statements)

References 20 publications

Supporting

Mentioning

243

Contrasting

Order By: Relevance

“…Factor VAE metric [16], Mutual Information Gap (MIG) [17], Modularity [19], DCI Disentanglement [20] and SAP score [14], similar to [21]. We show the disentanglement performance of the proposed method with the following existing variants of VAEs: 1) β-VAE, 2) FactorVAE, 3) TC-VAE, 4)DIP-VAE-I, 5)DIP-VAE-II, 6) Bottleneck-VAE in Table 2.…”

Section: Resultsmentioning

confidence: 99%

$β$-Annealed Variational Autoencoder for glitches

Sankarapandian,

Kulis

2021

Preprint

View full text Add to dashboard Cite

Gravitational wave detectors such as LIGO and Virgo are susceptible to various types of instrumental and environmental disturbances known as glitches which can mask and mimic gravitational waves. While there are 22 classes of non-Gaussian noise gradients currently identified, the number of classes is likely to increase as these detectors go through commissioning between observation runs. Since identification and labelling new noise gradients can be arduous and time-consuming, we propose β-Annelead VAEs to learn representations from spectograms in an unsupervised way. Using the same formulation as [1], we view Bottleneck-VAEs [2] through the lens of information theory and connect them to β-VAEs [3]. Motivated by this connection, we propose an annealing schedule for the hyperparameter β in β-VAEs which has advantages of: 1) One fewer hyperparameter to tune, 2) Better reconstruction quality, while producing similar levels of disentanglement.

show abstract

Section: Resultsmentioning

confidence: 99%

$β$-Annealed Variational Autoencoder for glitches

Sankarapandian,

Kulis

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In all single-stage methods, the disentanglement has improved due to the partial knowledge of label information through cross-entropy loss. Meanwhile, two-stage disentanglement methods, such as β-TCVAE [27], β-VAE [28], factor-VAE [29], and joint continuous-discrete factors VAE [30] methods create separate latent spaces without knowledge of class-label information. These methods mostly deal with autoencoder structures with a Gaussian prior distribution in the encoded space.…”

Section: A Disentangle Latent Space Clusteringmentioning

confidence: 99%

Improving ClusterGAN Using Self-AugmentedInformation Maximization of Disentangling LatentSpaces

Dam¹,

Anavatti²,

Abbass³

2021

Preprint

View full text Add to dashboard Cite

The Latent Space Clustering in Generative Adversarial Networks (ClusterGAN) method has been successful with high dimensional data. However, the method assumes uniformly distributed priors during the generation of modes, which is a restrictive assumption in real-world data and cause loss of diversity in the generated modes. In this paper, we propose self-augmentation information maximization improved Clus-terGAN (SIMI-ClusterGAN) to learn the distinctive priors from the data. The proposed SIMI-ClusterGAN consists of four deep neural networks: self-augmentation prior network, generator, discriminator and clustering inference autoencoder. The proposed method has been validated using seven benchmark data sets and has shown improved performance over state-of-the art methods. To demonstrate the superiority of SIMI-ClusterGAN performance on imbalanced dataset, we have discussed two imbalanced conditions on MNIST datasets with one-class imbalance and three classes imbalanced cases. The results highlight the advantages of SIMI-ClusterGAN.

show abstract

“…BetaVAE [14] uses a modified VAE objective that encourages the reduction of the KL-divergenece to an isotropic Gaussian prior to ensure disentanglement of all latent codes. FactorVAE [16] encourages factorization of the aggregate posterior by approximating the KL-Divergence using the cross-entropy loss of a classifier. Conditional VAE [27] improves the performance of VAE by conditioning the latent variable distribution over another variable.…”

Section: Related Workmentioning

confidence: 99%

“…We compare FRIED's performance with several baselines, namely, (i) FactorVAE [16], (ii) BetaVAE [14], (iii) CVAE [27], and (iv) Variational Fair AE (VFAE) [19]. The results for the UCI Adult and the dSprites datasets are shown in Fig.…”

Section: Fairness-accuracy Trade Offmentioning

confidence: 99%

Fair Representation Learning using Interpolation Enabled Disentanglement

Jha,

Vinzamuri,

Reddy

2021

Preprint

View full text Add to dashboard Cite

With the growing interest in machine learning community to solve real world problems, it has become crucial to uncover the hidden reasoning behind their decisions by focusing on the fairness and auditing the predictions made by these black-box models. In this paper, we propose a novel method to address two key issues: (a) Can we simultaneously learn fair disentangled representations while ensuring the utility of the learned representation for downstream tasks, and (b) Can we provide theoretical insights into when the proposed approach will be both fair and accurate. To address the former, we propose the method FRIED, Fair Representation learning using Interpolation Enabled Disentanglement. In our architecture, by imposing a critic-based adversarial framework, we enforce the interpolated points in the latent space to be more realistic. This helps in capturing the data manifold effectively and enhances utility of the learned representation for downstream prediction tasks. We address the latter question by developing theory on fairness-accuracy trade-offs using classifier-based conditional mutual information estimation. We demonstrate the effectiveness of FRIED on datasets of different modalities -tabular, text, and image datasets. We observe that the representations learned by FRIED are overall fairer in comparison to existing baselines and also accurate for downstream prediction tasks. Additionally, we evaluate FRIED on a real-world healthcare claims dataset where we conduct an expert aided model auditing study providing useful insights into opioid addiction patterns.

show abstract

Disentangling by Factorising

Cited by 131 publications

References 20 publications

$β$-Annealed Variational Autoencoder for glitches

$β$-Annealed Variational Autoencoder for glitches

Improving ClusterGAN Using Self-AugmentedInformation Maximization of Disentangling LatentSpaces

Fair Representation Learning using Interpolation Enabled Disentanglement

Contact Info

Product

Resources

About