Augmentation-Free Self-Supervised Learning on Graphs

Lee, Namkyeong; Lee, Junseok; Park, Chanyoung

doi:10.48550/arxiv.2112.02472

Cited by 3 publications

(9 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…2) Graph diffusion could achieve comparable performance on CORA and AMAP datasets while can not compare with ours on other datasets. It indicates that graph diffusion might change the underlying semantics of graphs [17]. Overall, expensive experiments have demonstrated the effectiveness of our proposed data augmentation method.…”

Section: Ablation Studiesmentioning

confidence: 72%

“…In summary, we construct two augmented views Z 𝑣 1 and Z 𝑣 2 by designing parameter un-shared encoders and corrupting the node embeddings directly instead of introducing complex operations against graphs, thus improving the training efficiency (See section 4.4). Besides, recent works [17,28,32] have indicated that the complex data augmentations over graphs, like edge adding, edge dropping, and graph diffusion could lead to semantic drift. The similar conclusion is verified through experiments in Section 4.5.2.…”

Section: Structural Contrastive Modulementioning

confidence: 99%

See 1 more Smart Citation

Simple Contrastive Graph Clustering

Liu¹,

Yang²,

Zhou³

et al. 2022

Preprint

View full text Add to dashboard Cite

Contrastive learning has recently attracted plenty of attention in deep graph clustering for its promising performance. However, complicated data augmentations and time-consuming graph convolutional operation undermine the efficiency of these methods.To solve this problem, we propose a Simple Contrastive Graph Clustering (SCGC) algorithm to improve the existing methods from the perspectives of network architecture, data augmentation, and objective function. As to the architecture, our network includes two main parts, i.e., pre-processing and network backbone. A simple low-pass denoising operation conducts neighbor information aggregation as an independent pre-processing, and only two multilayer perceptrons (MLPs) are included as the backbone. For data augmentation, instead of introducing complex operations over graphs, we construct two augmented views of the same vertex by designing parameter un-shared siamese encoders and corrupting the node embeddings directly. Finally, as to the objective function, to further improve the clustering performance, a novel cross-view structural consistency objective function is designed to enhance the discriminative capability of the learned network. Extensive experimental results on seven benchmark datasets validate our proposed algorithm's effectiveness and superiority. Significantly, our algorithm outperforms the recent contrastive deep clustering competitors with at least seven times speedup on average. CCS CONCEPTS• Computing methodologies → Cluster analysis.

show abstract

Section: Ablation Studiesmentioning

confidence: 72%

Section: Structural Contrastive Modulementioning

confidence: 99%

Simple Contrastive Graph Clustering

Liu¹,

Yang²,

Zhou³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…However, due to the natural complexity of graphs, the mentioned SSL-based model highly depends on the data augmentation scheme [11], and [18] has shown that there is no universally outperforming data augmentation scheme for graphs. AFGRL [11] first proposes to generate positive pairs based on the local structural information and the global semantics of graphs instead of data augmentation, where the structure of the original graph will not be destroyed, thus preserving the semantics. Simultaneously, SimGCL [9] discards the augmentation and creates contrastive positives by adding uniform noises to the embedding space.…”

Section: Augmentations On Graphsmentioning

confidence: 99%

“…However, the Unfiltered i A contains a lot of noise nodes, namely "false" positives that are semantically contrary to the target node and will lead to poor performance when regarded as positives. Therefore, to filter out "false" positives in the neighboring nodes, following [11], we then compute the cosine similarity between all other nodes in the graph as follows:…”

Section: Mixing Layermentioning

confidence: 99%

See 1 more Smart Citation

Recommend What to Cache: a Simple Self-supervised Graph-based Recommendation Framework for Edge Caching Network

Sun

Wang

Han³

2023

Preprint

View full text Add to dashboard Cite

The edge caching network based on deep learning can more accurately infer what to cache while driven by data from the user's historical content request, thus greatly relieving the burden of the backbone network. However, the inherent cold-start problem in deep learning may limit the performance of the history-based caching strategies. Due to the mobile and dynamic nature of the wireless network, the base station usually does not have enough data to accurately estimate the user's demand and cache the possible requested data. In this backdrop, we adopt self-supervised learning (SSL) into the caching strategy and propose a Simple Self-supervised Graph-based Recommendation framework for edge caching network (SimSGR). The SSL could alleviate the cold-start problem by generating extra training supervised signals from the raw data. To further optimize the performance of SSL, we propose two new network layers: Mixing and Conversion: the former replaces the data augmentation of the SSL paradigm to avoid destroying the semantic loss, while the latter greatly simplifies the loss function, which is helpful to light the model structure and facilitate deployment on edge caching network. Extensive experiments show that our model yields competitive or better performance than state-of-the-art methods sensitive to augmentation hyperparameters, especially when trained in the cold-start environment.

show abstract

Cluster-Guided Contrastive Graph Clustering Network

Yang

Liu

Zhou

et al. 2023

AAAI

View full text Add to dashboard Cite

Benefiting from the intrinsic supervision information exploitation capability, contrastive learning has achieved promising performance in the field of deep graph clustering recently. However, we observe that two drawbacks of the positive and negative sample construction mechanisms limit the performance of existing algorithms from further improvement. 1) The quality of positive samples heavily depends on the carefully designed data augmentations, while inappropriate data augmentations would easily lead to the semantic drift and indiscriminative positive samples. 2) The constructed negative samples are not reliable for ignoring important clustering information. To solve these problems, we propose a Cluster-guided Contrastive deep Graph Clustering network (CCGC) by mining the intrinsic supervision information in the high-confidence clustering results. Specifically, instead of conducting complex node or edge perturbation, we construct two views of the graph by designing special Siamese encoders whose weights are not shared between the sibling sub-networks. Then, guided by the high-confidence clustering information, we carefully select and construct the positive samples from the same high-confidence cluster in two views. Moreover, to construct semantic meaningful negative sample pairs, we regard the centers of different high-confidence clusters as negative samples, thus improving the discriminative capability and reliability of the constructed sample pairs. Lastly, we design an objective function to pull close the samples from the same cluster while pushing away those from other clusters by maximizing and minimizing the cross-view cosine similarity between positive and negative samples. Extensive experimental results on six datasets demonstrate the effectiveness of CCGC compared with the existing state-of-the-art algorithms. The code of CCGC is available at https://github.com/xihongyang1999/CCGC on Github.

show abstract

Augmentation-Free Self-Supervised Learning on Graphs

Cited by 3 publications

References 31 publications

Simple Contrastive Graph Clustering

Simple Contrastive Graph Clustering

Recommend What to Cache: a Simple Self-supervised Graph-based Recommendation Framework for Edge Caching Network

Cluster-Guided Contrastive Graph Clustering Network

Contact Info

Product

Resources

About