Shared Generative Latent Representation Learning for Multi-View Clustering

Yin, Ming; Huang, Weitian; Gao, Junbin

doi:10.1609/aaai.v34i04.6146

Cited by 44 publications

(16 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Yang et al [48] proposed graph embedding in a Gaussian mixture variational autoencoder. Although there are already some VAE-based multi-view or multi-modal learning methods, such as [8,20,42,50], our work is the first attempt to give a disentangled multi-view VAE framework in view-common and view-peculiar representation learning perspectives.…”

Section: Related Workmentioning

confidence: 99%

“…For many MVC methods, the central bottleneck is their high complexity that makes it unrealistic for handling large-scale data clustering tasks. Recent approaches have achieved inspirational progress by applying deep models [3,7,34,45,50,55]. However, most of them learn the clustering structure by exploring common representation or fusing features of all views.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multi-VAE: Learning Disentangled View-common and View-peculiar Visual Representations for Multi-view Clustering

Xu¹,

Ren²,

Tang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Multi-view clustering, a long-standing and important research problem, focuses on mining complementary information from diverse views. However, existing works often fuse multiple views' representations or handle clustering in a common feature space, which may result in their entanglement especially for visual representations. To address this issue, we present a novel VAE-based multi-view clustering framework (Multi-VAE) by learning disentangled visual representations. Concretely, we define a view-common variable and multiple view-peculiar variables in the generative model. The prior of view-common variable obeys approximately discrete Gumbel Softmax distribution, which is introduced to extract the common cluster factor of multiple views. Meanwhile, the prior of view-peculiar variable follows continuous Gaussian distribution, which is used to represent each view's peculiar visual factors. By controlling the mutual information capacity to disentangle the viewcommon and view-peculiar representations, continuous visual information of multiple views can be separated so that their common discrete cluster information can be effectively mined. Experimental results demonstrate that Multi-VAE enjoys the disentangled and explainable visual representations, while obtaining superior clustering performance compared with state-of-the-art methods.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Multi-VAE: Learning Disentangled View-common and View-peculiar Visual Representations for Multi-view Clustering

Xu¹,

Ren²,

Tang³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Some multimodal learning approaches learn a common low-dimensional latent space from different modalities. For example, variational autoencoder (VAE) neural networks are used to learn a shared latent representation of multiple modalities (Yin et al, 2020). Several Bayesian inference methods use the ability of exploring the spatial and temporal structures of data to fuse the modalities into a joint representation (Huang and Kingsbury, 2013).…”

Section: Related Workmentioning

confidence: 99%

Cross-Modal and Multimodal Data Analysis Based on Functional Mapping of Spectral Descriptors and Manifold Regularization

Behmanesh,

Adibi,

Chanussot

et al. 2021

Preprint

View full text Add to dashboard Cite

Multimodal manifold modeling methods extend the spectral geometry-aware data analysis to learning from several related and complementary modalities. Most of these methods work based on two major assumptions: 1) there are the same number of homogeneous data samples in each modality, and 2) at least partial correspondences between modalities are given in advance as prior knowledge. This work proposes two new multimodal modeling methods. The first method establishes a general analyzing framework to deal with the multimodal information problem for heterogeneous data without any specific prior knowledge. For this purpose, first, we identify the localities of each manifold by extracting local descriptors via spectral graph wavelet signatures (SGWS). Then, we propose a manifold regularization framework based on the functional mapping between SGWS descriptors (FMBSD) for finding the pointwise correspondences. The second method is a manifold regularized multimodal classification based on pointwise correspondences (M 2 CPC) used for the problem of multiclass classification of multimodal heterogeneous, which the correspondences between modalities are determined based on the FMBSD method. The experimental results of evaluating the FMBSD method on three common cross-modal retrieval datasets and evaluating the (M 2 CPC) method on three benchmark multimodal multiclass classification datasets indicate their effectiveness and superiority over state-of-the-art methods.

show abstract

“…The essential idea is to find several low-dimensional representations embedded in latent spaces and finally attain a united representation for downstream clustering tasks [ 26 ]. Besides, aiming at finding a shared low-dimensional latent representation via matrix decomposition, the non-negative matrix factorization (NMF) [ 27 ]-based multi-view clustering methods [ 28 , 29 , 30 , 31 ] can also be seen as a branch of the subspace-based multi-view clustering method.…”

Section: Introductionmentioning

confidence: 99%

Adaptive Weighted Graph Fusion Incomplete Multi-View Subspace Clustering

Zhang

Wang

et al. 2020

Sensors

View full text Add to dashboard Cite

With the enormous amount of multi-source data produced by various sensors and feature extraction approaches, multi-view clustering (MVC) has attracted developing research attention and is widely exploited in data analysis. Most of the existing multi-view clustering methods hold on the assumption that all of the views are complete. However, in many real scenarios, multi-view data are often incomplete for many reasons, e.g., hardware failure or incomplete data collection. In this paper, we propose an adaptive weighted graph fusion incomplete multi-view subspace clustering (AWGF-IMSC) method to solve the incomplete multi-view clustering problem. Firstly, to eliminate the noise existing in the original space, we transform complete original data into latent representations which contribute to better graph construction for each view. Then, we incorporate feature extraction and incomplete graph fusion into a unified framework, whereas two processes can negotiate with each other, serving for graph learning tasks. A sparse regularization is imposed on the complete graph to make it more robust to the view-inconsistency. Besides, the importance of different views is automatically learned, further guiding the construction of the complete graph. An effective iterative algorithm is proposed to solve the resulting optimization problem with convergence. Compared with the existing state-of-the-art methods, the experiment results on several real-world datasets demonstrate the effectiveness and advancement of our proposed method.

show abstract

Shared Generative Latent Representation Learning for Multi-View Clustering

Cited by 44 publications

References 18 publications

Multi-VAE: Learning Disentangled View-common and View-peculiar Visual Representations for Multi-view Clustering

Multi-VAE: Learning Disentangled View-common and View-peculiar Visual Representations for Multi-view Clustering

Cross-Modal and Multimodal Data Analysis Based on Functional Mapping of Spectral Descriptors and Manifold Regularization

Adaptive Weighted Graph Fusion Incomplete Multi-View Subspace Clustering

Contact Info

Product

Resources

About