Strongly augmented contrastive clustering

Deng, Xiaozhi; Huang, Dong; Chen, Ding-Hua; Wang, Chang‐Dong; Lai, Jianhuang

doi:10.1016/j.patcog.2023.109470

Cited by 37 publications

(9 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…MoCov2 [49] improves MoCo by using a multi-layer perceptron (MLP) projection head and more data augmentations to ease the burden on the batch size in training. SACC [52] incorporates strong and weak augmentations into instance-and cluster-level CL for deep clustering. Moreover, there are many recent methods extending CL to graph domains [53]- [59].…”

Section: Contrastive Learningmentioning

confidence: 99%

Towards Long-Tailed Recognition for Graph Classification via Collaborative Experts

Yi,

Mao,

et al. 2023

IEEE Trans. Big Data

View full text Add to dashboard Cite

Graph classification, aiming at learning the graphlevel representations for effective class assignments, has received outstanding achievements, which heavily relies on high-quality datasets that have balanced class distribution. In fact, most real-world graph data naturally presents a long-tailed form, where the head classes occupy much more samples than the tail classes, it thus is essential to study the graph-level classification over long-tailed data while still remaining largely unexplored. However, most existing long-tailed learning methods in visions fail to jointly optimize the representation learning and classifier training, as well as neglect the mining of the hard-to-classify classes. Directly applying existing methods to graphs may lead to sub-optimal performance, since the model trained on graphs would be more sensitive to the long-tailed distribution due to the complex topological characteristics. Hence, in this paper, we propose a novel long-tailed graph-level classification framework via Collaborative Multi-expert Learning (CoMe) to tackle the problem. To equilibrate the contributions of head and tail classes, we first develop balanced contrastive learning from the view of representation learning, and then design an individual-expert classifier training based on hard class mining. In addition, we execute gated fusion and disentangled knowledge distillation among the multiple experts to promote the collaboration in a multiexpert framework. Comprehensive experiments are performed on seven widely-used benchmark datasets to demonstrate the superiority of our method CoMe over state-of-the-art baselines.

show abstract

Section: Contrastive Learningmentioning

confidence: 99%

Towards Long-Tailed Recognition for Graph Classification via Collaborative Experts

Yi,

Mao,

et al. 2023

IEEE Trans. Big Data

View full text Add to dashboard Cite

show abstract

“…In terms of data augmentations, we take the same augmentations as SimCLR [1], which consists of ResizedCrop, ColorJitter, Grayscale and HorizontalFlip. However, GaussianBlur has been removed since we adopt only a small image size for all datasets [7], [10], [13]. In terms of mini-batch, the batch size is fixed at 256 for CIFAR-10 and CIFAR-100, and 128 for the rest of the datasets.…”

Section: A Implementation Detailsmentioning

confidence: 99%

“…Recently, deep clustering has made significant progress with the assistance of contrastive learning. Some deep clustering methods based on contrastive learning have achieved remarkable performance [7], [8], [10], [12], [13]. Specifically, [8] presented the IDFD method to learn similarities among instances and reduce correlations within features by adopting the idea of instance discrimination [27] and spectral clustering [19].…”

Section: A Deep Clusteringmentioning

confidence: 99%

“…Especially, [13] expanded the dual-branch structure of CC to a triple-branch structure and proposed a tri-stream clustering network called HTCN. Besides, the work presented in the SACC [10] explored the impact of strong and weak augmentations on contrastive clustering. [12] proposed a clustering framework with prototype scattering loss and positive sampling alignment, termed ProPos, which enjoys the strengths of contrastive and non-contrastive learning.…”

Section: A Deep Clusteringmentioning

confidence: 99%

“…In recent years, the contrastive learning has become a predominant technique in many deep clustering methods [6]- [10]. Specifically, these existing deep clustering methods are often built upon some classical contrastive learning models like SimCLR [1], which requires a symmetric (weight-sharing) network architecture and a well-designed loss function.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Deep Multiview Clustering via Iteratively Self-Supervised Universal and Specific Space Learning

Zhang

Huang

Zhang

et al. 2022

IEEE Trans. Cybern.

View full text Add to dashboard Cite

Deep clustering has gained significant attention due to its capability in learning clustering-friendly representations without labeled data. However, previous deep clustering methods tend to treat all samples equally, which neglect the variance in the latent distribution and the varying difficulty in classifying or clustering different samples. To address this, this paper proposes a novel end-to-end deep clustering method with diffused sampling and hardness-aware self-distillation (HaDis). Specifically, we first align one view of instances with another view via diffused sampling alignment (DSA), which helps improve the intra-cluster compactness. To alleviate the sampling bias, we present the hardness-aware self-distillation (HSD) mechanism to mine the hardest positive and negative samples and adaptively adjust their weights in a self-distillation fashion, which is able to deal with the potential imbalance in sample contributions during optimization. Further, the prototypical contrastive learning is incorporated to simultaneously enhance the inter-cluster separability and intracluster compactness. Experimental results on five challenging image datasets demonstrate the superior clustering performance of our HaDis method over the state-of-the-art. Source code is available at https://github.com/Regan-Zhang/HaDis.

show abstract

Learning approaches and tricks

Ros,

Riad

2023

Unsupervised and Semi-Supervised Learning

View full text Add to dashboard Cite

Strongly augmented contrastive clustering

Cited by 37 publications

References 14 publications

Towards Long-Tailed Recognition for Graph Classification via Collaborative Experts

Towards Long-Tailed Recognition for Graph Classification via Collaborative Experts

Deep Multiview Clustering via Iteratively Self-Supervised Universal and Specific Space Learning

Learning approaches and tricks

Contact Info

Product

Resources

About