Unsupervised Clustering through Gaussian Mixture Variational AutoEncoder with Non-Reparameterized Variational Inference and Std Annealing

Zhao, Youjian; Xu, Haowen; Xu, Shangqing; Li, Yilin; Pei, Dan

doi:10.1109/ijcnn48605.2020.9207493

Cited by 10 publications

(2 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An alternative simple migration [85] yields a composition of an InfoNCE loss [61] and a clustering one [77]. Compared with the deep generative counterparts [16,36,53,82], contrastive clustering is free from decoding and computationally practical, with guaranteed feature quality.…”

Section: Introductionmentioning

confidence: 99%

You Never Cluster Alone

Shen¹,

Shen²,

Wang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Recent advances in self-supervised learning with instance-level contrastive objectives facilitate unsupervised clustering. However, a standalone datum is not perceiving the context of the holistic cluster, and may undergo sub-optimal assignment. In this paper, we extend the mainstream contrastive learning paradigm to a cluster-level scheme, where all the data subjected to the same cluster contribute to a unified representation that encodes the context of each data group. Contrastive learning with this representation then rewards the assignment of each datum. To implement this vision, we propose twin-contrast clustering (TCC). We define a set of categorical variables as clustering assignment confidence, which links the instancelevel learning track with the cluster-level one. On one hand, with the corresponding assignment variables being the weight, a weighted aggregation along the data points implements the set representation of a cluster. We further propose heuristic cluster augmentation equivalents to enable cluster-level contrastive learning. On the other hand, we derive the evidence lower-bound of the instance-level contrastive objective with the assignments. By reparametrizing the assignment variables, TCC is trained end-to-end, requiring no alternating steps. Extensive experiments show that TCC outperforms the state-of-the-art on challenging benchmarks.Preprint. Under review.

show abstract

Section: Introductionmentioning

confidence: 99%

You Never Cluster Alone

Shen¹,

Shen²,

Wang³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Although VAE can learn data distributions in the latent space which is good for unsupervised learning tasks, the Gaussian prior may lead to crowding clusters, hindering the subsequent clustering process to effectively separate different groups [34]. Several DC models [35][36][37][38] turn to the Gaussian mixture prior for modeling the discrete clusters in the latent space, which have contributed to a more clustering-oriented VAE framework. In spite of that, vanilla VAE lacks the ability that captures the detailed features of objects and is notorious for blurry output images [39][40][41], which could potentially lead to inferior clustering performance.…”

Section: Introductionmentioning

confidence: 99%