Multi-VAE: Learning Disentangled View-common and View-peculiar Visual Representations for Multi-view Clustering

Xu, Jie; Ren, Yazhou; Tang, Huayi; Pu, Xiaorong; Zhu, Xiaofeng; Zeng, Mingyong; He, Lifang

doi:10.1109/iccv48922.2021.00910

Cited by 70 publications

(30 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In Section 3.1, we analyse how FA-VAE is able to condition a pretrained VAE to multilabel targets. In Section 3.2, we apply it over a domain adaptation problem and compare it to the Multi-VAE model [21], besides, we disentangle and analyse the private-shared latent variables. Finally, in Section 3.3 we use the proposed FA-VAE's framework to perform transfer learning between multiple VAEs, showing how the transfer learning creates a more expressive and understandable latent space than other models such as β-VAE [30].…”

Section: Methodsmentioning

confidence: 99%

“…x (D) n,: We compare FA-VAE with the Multi-VAE [21] model which uses a c n discrete latent variable to share the context of all views (see Fig. 8).…”

Section: Domain Adaptationmentioning

confidence: 99%

“…Following this idea, in AMVAE [20], the K views are projected into a common latent space but using a fully-connected layer. Yet, in Multi-VAE [21], the authors propose a probabilistic change, since each view is represented by a private continuous latent variable and the shared information is modeled by a categorical discrete latent variable by means of a Gumbel softmax [22] reparametrization. Some of these above mentioned models can also perform domain adaption between different datasets.…”

Section: Introductionmentioning

confidence: 99%

“…For example, MVAE [19] performs machine translation between English and Vietnamese by training a VAE for each language. The Multi-VAE [21] is able to generate new samples of MNIST [23] by conditioning the images to their writer style. Moreover, UGVAE [7] performs domain image adaptation between the datasets CelebA [24] and 3D-Faces [25].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Multimodal hierarchical Variational AutoEncoders with Factor Analysis latent space

Guerrero-López¹,

Sevilla-Salcedo²,

Gómez-Verdejo³

et al. 2022

Preprint

View full text Add to dashboard Cite

Real-world databases are complex, they usually present redundancy and shared correlations between heterogeneous and multiple representations of the same data. Thus, exploiting and disentangling shared information between views is critical. For this purpose, recent studies often fuse all views into a shared nonlinear complex latent space but they lose the interpretability. To overcome this limitation, here we propose a novel method to combine multiple Variational AutoEncoders (VAE) architectures with a Factor Analysis latent space (FA-VAE).Concretely, we use a VAE to learn a private representation of each heterogeneous view in a continuous latent space. Then, we model the shared latent space by projecting every private variable to a low-dimensional latent space using a linear projection matrix. Thus, we create an interpretable hierarchical dependency between private and shared information. This way, the novel model is able to simultaneously: (i) learn from multiple heterogeneous views, (ii) obtain an interpretable hierarchical shared space, and, (iii) perform transfer learning between generative models.

show abstract

Section: Methodsmentioning

confidence: 99%

“…x (D) n,: We compare FA-VAE with the Multi-VAE [21] model which uses a c n discrete latent variable to share the context of all views (see Fig. 8).…”

Section: Domain Adaptationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Multimodal hierarchical Variational AutoEncoders with Factor Analysis latent space

Guerrero-López¹,

Sevilla-Salcedo²,

Gómez-Verdejo³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Multi-view clustering is an important unsupervised approach, aiming to improve the model effectiveness by mining the complementary information hidden in multi-view data. Recently, many multi-view clustering methods have been proposed (Tao et al 2017;Li et al 2019;Peng et al 2019;Tang et al 2020;Chen et al 2020;Huang et al 2021;Xu et al 2021b). These methods mainly deal with the complete multi-view data, where the information of all views is observed.…”

Section: Introductionmentioning

confidence: 99%

Deep Incomplete Multi-View Clustering via Mining Cluster Complementarity

Ren

et al. 2022

AAAI

Self Cite

View full text Add to dashboard Cite

Incomplete multi-view clustering (IMVC) is an important unsupervised approach to group the multi-view data containing missing data in some views. Previous IMVC methods suffer from the following issues: (1) the inaccurate imputation or padding for missing data negatively affects the clustering performance, (2) the quality of features after fusion might be interfered by the low-quality views, especially the inaccurate imputed views. To avoid these issues, this work presents an imputation-free and fusion-free deep IMVC framework. First, the proposed method builds a deep embedding feature learning and clustering model for each view individually. Our method then nonlinearly maps the embedding features of complete data into a high-dimensional space to discover linear separability. Concretely, this paper provides an implementation of the high-dimensional mapping as well as shows the mechanism to mine the multi-view cluster complementarity. This complementary information is then transformed to the supervised information with high confidence, aiming to achieve the multi-view clustering consistency for the complete data and incomplete data. Furthermore, we design an EM-like optimization strategy to alternately promote feature learning and clustering. Extensive experiments on real-world multi-view datasets demonstrate that our method achieves superior clustering performance over state-of-the-art methods.

show abstract

An Exploration of the Latent Space of a Convolutional Variational Autoencoder for the Generation of Musical Instrument Tones

Natsiou,

O’Leary,

Longo

2023

Communications in Computer and Information Science

View full text Add to dashboard Cite

Multi-VAE: Learning Disentangled View-common and View-peculiar Visual Representations for Multi-view Clustering

Cited by 70 publications

References 20 publications

Multimodal hierarchical Variational AutoEncoders with Factor Analysis latent space

Multimodal hierarchical Variational AutoEncoders with Factor Analysis latent space

Deep Incomplete Multi-View Clustering via Mining Cluster Complementarity

An Exploration of the Latent Space of a Convolutional Variational Autoencoder for the Generation of Musical Instrument Tones

Contact Info

Product

Resources

About