Graph Contrastive Clustering

Zhong, Huasong; Wu, Jian; Chen, Chong; Huang, Jianqiang; Deng, Minghua; Nie, Liqiang; Lin, Zhouchen; Hua, Xian-Sheng

doi:10.1109/iccv48922.2021.00909

Cited by 83 publications

(32 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, SCAN [6] yields the confident pseudo-labels by the pre-trained SimCLR model, and IDFD [9] proposes to perform both instance discrimination and feature decorrelation. Although GCC [41] and WCL [42] select the neighbor samples from a graph as pseudo-positive examples for contrastive loss, however, they still suffer from the class collision issue as these selected examples may not be truly positive. In a nutshell, all of them are built upon the contrastive learning framework, in which they require a large number of negative examples to maintain uniform representations, inevitably leading to class collision issue.…”

Section: Deep Clusteringmentioning

confidence: 99%

“…In Sec. 6, we discuss the differences from existing methods including CC [8], GCC [41], WCL [42], PCL [7], and instance-reweighted contrastive loss [45].…”

Section: Deep Clusteringmentioning

confidence: 99%

“…Therefore, optimizing PSA loss would not cause class collision issue that exists in contrastive-based deep clustering methods. We discuss the difference from [41], [42] that sample positive examples from the dataset in Sec. 6.…”

Section: Positive Sampling Alignmentmentioning

confidence: 99%

“…We emphasize that it is desirable to avoid the trivial solution for deep clustering. For example, CC [8] and etc [41], [53] have usually employed an entropy term in loss function to regularize the model equally assigning the images into different clusters. This paper mainly investigates this phenomenon of non-contrastive representation learning for deep clustering, and the theoretical analysis can be found in [14], [54].…”

Section: The Role Of Pslmentioning

confidence: 99%

See 3 more Smart Citations

Learning Representation for Clustering Via Prototype Scattering and Positive Sampling

Huang

Chen

Zhang

et al. 2023

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Existing deep clustering methods rely on either contrastive or non-contrastive representation learning for downstream clustering task. Contrastive-based methods thanks to negative pairs learn uniform representations for clustering, in which negative pairs, however, may inevitably lead to the class collision issue and consequently compromise the clustering performance. Non-contrastive-based methods, on the other hand, avoid class collision issue, but the resulting non-uniform representations may cause the collapse of clustering. To enjoy the strengths of both worlds, this paper presents a novel end-to-end deep clustering method with prototype scattering and positive sampling, termed ProPos. Specifically, we first maximize the distance between prototypical representations, named prototype scattering loss, which improves the uniformity of representations. Second, we align one augmented view of instance with the sampled neighbors of another view-assumed to be truly positive pair in the embedding space-to improve the within-cluster compactness, termed positive sampling alignment. The strengths of ProPos are avoidable class collision issue, uniform representations, well-separated clusters, and within-cluster compactness. By optimizing ProPos in an end-to-end expectation-maximization framework, extensive experimental results demonstrate that ProPos achieves competing performance on moderate-scale clustering benchmark datasets and establishes new state-of-the-art performance on large-scale datasets. Source code is available at https://github.com/Hzzone/ProPos.

show abstract

Section: Deep Clusteringmentioning

confidence: 99%

“…In Sec. 6, we discuss the differences from existing methods including CC [8], GCC [41], WCL [42], PCL [7], and instance-reweighted contrastive loss [45].…”

Section: Deep Clusteringmentioning

confidence: 99%

Section: Positive Sampling Alignmentmentioning

confidence: 99%

Section: The Role Of Pslmentioning

confidence: 99%

See 2 more Smart Citations

Learning Representation for Clustering Via Prototype Scattering and Positive Sampling

Huang

Chen

Zhang

et al. 2023

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

show abstract

“…Contrastive learning has achieved great success on images [3,7,12,51,53,56] and graphs [1,13,31,38,45,52,58] in recent years. Inspired by their success, contrastive deep graph clustering methods [5, 11, 20-22, 46, 47, 55] are increasingly proposed.…”

Section: Contrastive Deep Graph Clusteringmentioning

confidence: 99%

Simple Contrastive Graph Clustering

Liu¹,

Yang²,

Zhou³

et al. 2022

Preprint

View full text Add to dashboard Cite

Contrastive learning has recently attracted plenty of attention in deep graph clustering for its promising performance. However, complicated data augmentations and time-consuming graph convolutional operation undermine the efficiency of these methods.To solve this problem, we propose a Simple Contrastive Graph Clustering (SCGC) algorithm to improve the existing methods from the perspectives of network architecture, data augmentation, and objective function. As to the architecture, our network includes two main parts, i.e., pre-processing and network backbone. A simple low-pass denoising operation conducts neighbor information aggregation as an independent pre-processing, and only two multilayer perceptrons (MLPs) are included as the backbone. For data augmentation, instead of introducing complex operations over graphs, we construct two augmented views of the same vertex by designing parameter un-shared siamese encoders and corrupting the node embeddings directly. Finally, as to the objective function, to further improve the clustering performance, a novel cross-view structural consistency objective function is designed to enhance the discriminative capability of the learned network. Extensive experimental results on seven benchmark datasets validate our proposed algorithm's effectiveness and superiority. Significantly, our algorithm outperforms the recent contrastive deep clustering competitors with at least seven times speedup on average. CCS CONCEPTS• Computing methodologies → Cluster analysis.

show abstract