Twin Contrastive Learning for Online Clustering

Li, Yunfan; Yang, Mouxing; Peng, Dezhong; Li, Taihao; Huang, Jian; Peng, Xi

doi:10.1007/s11263-022-01639-z

Cited by 77 publications

(19 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, our ProPos has introduced further substantial improvements for deep clustering by addressing existing issues. Although CC [8] and TCL [38] achieve the better NMIs on STL-10, we highlight that they use a large image size of 224 × 224 for all datasets, which is not fair to ours.…”

Section: Methods Amicontrasting

confidence: 58%

“…Deep clustering [30], [31], [32], [33], [34], [35], [36], [37], [38] has been significantly advanced by self-supervised representation learning. Most of deep clustering methods are based on contrastive learning by exploiting the discriminative representations, learned from contrastive learning, to assist the downstream clustering tasks [6], [11], [38] or simultaneously optimize representation learning and clustering [7], [9], [10], [39], [40]. For example, SCAN [6] yields the confident pseudo-labels by the pre-trained SimCLR model, and IDFD [9] proposes to perform both instance discrimination and feature decorrelation.…”

Section: Deep Clusteringmentioning

confidence: 99%

See 1 more Smart Citation

Learning Representation for Clustering Via Prototype Scattering and Positive Sampling

Huang

Chen

Zhang

et al. 2023

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Existing deep clustering methods rely on either contrastive or non-contrastive representation learning for downstream clustering task. Contrastive-based methods thanks to negative pairs learn uniform representations for clustering, in which negative pairs, however, may inevitably lead to the class collision issue and consequently compromise the clustering performance. Non-contrastive-based methods, on the other hand, avoid class collision issue, but the resulting non-uniform representations may cause the collapse of clustering. To enjoy the strengths of both worlds, this paper presents a novel end-to-end deep clustering method with prototype scattering and positive sampling, termed ProPos. Specifically, we first maximize the distance between prototypical representations, named prototype scattering loss, which improves the uniformity of representations. Second, we align one augmented view of instance with the sampled neighbors of another view-assumed to be truly positive pair in the embedding space-to improve the within-cluster compactness, termed positive sampling alignment. The strengths of ProPos are avoidable class collision issue, uniform representations, well-separated clusters, and within-cluster compactness. By optimizing ProPos in an end-to-end expectation-maximization framework, extensive experimental results demonstrate that ProPos achieves competing performance on moderate-scale clustering benchmark datasets and establishes new state-of-the-art performance on large-scale datasets. Source code is available at https://github.com/Hzzone/ProPos.

show abstract

Section: Methods Amicontrasting

confidence: 58%

Section: Deep Clusteringmentioning

confidence: 99%

Learning Representation for Clustering Via Prototype Scattering and Positive Sampling

Huang

Chen

Zhang

et al. 2023

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

show abstract

“…Several strategies, largely based on clustering techniques, have been developed to avoid substantial job-specific feature engineering because of the dramatically increased size of the datasets. There are many research works and examples of pure advanced clustering methods [ 30 , 31 , 32 , 33 ] to cluster a dataset. Yunfan Li et al [ 30 ] recommend a one-stage online clustering method that directly generates positive and negative instance pairs using data augmentation and afterwards projects the pairs in a feature space.…”

Section: Literature Reviewmentioning

confidence: 99%

“…Moreover, the research in [ 47 ] provided a semantic tagging strategy that makes use of Wikipedia’s knowledge to methodically identify content for social software engineering while also semantically grounding the tagging process. Despite the availability of advanced clustering methods [ 30 , 31 , 32 , 33 ], we aimed to show contribution of clustering technique with basic k-means algorithm to improve additionally the effectiveness of hierarchically organized hidden implicit features of customers and products in building a recommendation model.…”

Section: Literature Reviewmentioning

confidence: 99%

“…The following are some directions for further research: To design a recommender system that is understandable and comprehensible using implicit hidden characteristics; To use metaheuristic techniques to enhance performance metrics [ 57 ]; To handle the “grey sheep” issue, which occurs when a customer cannot be matched with any other customer group, and the system is unable to produce helpful recommendations [ 58 ]; To provide dynamic predictions with the least amount of complexity; To develop an emotion-based movie recommendation model [ 59 , 60 ]; To integrate other advanced clustering methods such as twin contrastive learning for online clustering, structured autoencoders for subspace clustering, and XAI beyond classification: interpretable neural clustering [ 30 , 31 , 32 , 33 ] for further models’ improvement and to analyze clustering techniques contribution. …”

Section: Conclusion and Future Scopementioning

confidence: 99%

See 1 more Smart Citation

Modeling and Applying Implicit Dormant Features for Recommendation via Clustering and Deep Factorization

Kutlimuratov

Abdusalomov

Oteniyazov

et al. 2022

Sensors

View full text Add to dashboard Cite

E-commerce systems experience poor quality of performance when the number of records in the customer database increases due to the gradual growth of customers and products. Applying implicit hidden features into the recommender system (RS) plays an important role in enhancing its performance due to the original dataset’s sparseness. In particular, we can comprehend the relationship between products and customers by analyzing the hierarchically expressed hidden implicit features of them. Furthermore, the effectiveness of rating prediction and system customization increases when the customer-added tag information is combined with hierarchically structured hidden implicit features. For these reasons, we concentrate on early grouping of comparable customers using the clustering technique as a first step, and then, we further enhance the efficacy of recommendations by obtaining implicit hidden features and combining them via customer’s tag information, which regularizes the deep-factorization procedure. The idea behind the proposed method was to cluster customers early via a customer rating matrix and deeply factorize a basic WNMF (weighted nonnegative matrix factorization) model to generate customers preference’s hierarchically structured hidden implicit features and product characteristics in each cluster, which reveals a deep relationship between them and regularizes the prediction procedure via an auxiliary parameter (tag information). The testimonies and empirical findings supported the viability of the proposed approach. Especially, MAE of the rating prediction was 0.8011 with 60% training dataset size, while the error rate was equal to 0.7965 with 80% training dataset size. Moreover, MAE rates were 0.8781 and 0.9046 in new 50 and 100 customer cold-start scenarios, respectively. The proposed model outperformed other baseline models that independently employed the major properties of customers, products, or tags in the prediction process.

show abstract