Modeling Multiple Views via Implicitly Preserving Global Consistency and Local Complementarity

Abstract

While self-supervised learning techniques are often used to mine hidden knowledge from unlabeled data via modeling multiple views, it is unclear how to perform effective representation learning in a complex and inconsistent context. To this end, we propose a new multi-view self-supervised learning method, namely consistency and complementarity network (CoCoNet), to comprehensively learn global inter-view consistent and local cross-view complementarity-preserving representations from multiple views. To capture crucial common knowledge which is implicitly shared among views, CoCoNet employs a global consistency module that aligns the probabilistic distribution of views by utilizing an efficient discrepancy metric based on the generalized sliced Wasserstein distance. To incorporate cross-view complementary information, CoCoNet proposes a heuristic complementarity-aware contrastive learning approach, which extracts a complementarity-factor jointing cross-view discriminative knowledge and uses it as the contrast to guide the learning of view-specific encoders. Theoretically, the superiority of CoCoNet is verified by our information-theoretical-based analyses. Empirically, our thorough experimental results show that CoCoNet outperforms the state-of-the-art self-supervised methods by a significant margin, for instance, CoCoNet beats the best benchmark method by an average margin of 1.1% on ImageNet.

Hierarchical Topology Isomorphism Expertise Embedded Graph Contrastive Learning

“…As one of the most effective self-supervised methods, contrastive learning (CL) aims to learn discriminative representations from unlabelled data (Li et al 2022a). With the principle of closing positive and moving away from negative pairs, CL methods, such as SimCLR (Chen et al 2020), MoCo (He et al 2020), BYOL (Grill et al 2020), MetAug (Li et al 2022b), andBarlow Twins (Zbontar et al 2021), have achieved outstanding success in the field of computer vision (Wu et al 2018;Chen et al 2020;Qiang et al 2022).…”

MetaMask: Revisiting Dimensional Confounder for Self-Supervised Learning

“…A fundamental idea behind self-supervised learning is to learn discriminative representations from the input data without relying on human annotations. Recent advances in visual self-supervised learning [3,4,5,6,7] demonstrate that unsupervised approaches can achieve competitive performance over supervised approaches by introducing sophisticated self-supervised tasks. A representative learning paradigm is contrastive learning [8,1,9,10,11,12,13], which aims to learn invariant information from different views (generated by data augmentations) by performing instance-level contrast, i.e., pulling views of the same sample together while pushing views of different samples away.…”