Adversarial Graph Contrastive Learning with Information Regularization

Feng, Shengyu; Jing, Baoyu; Zhu, Yada; Tong, Hanghang

doi:10.1145/3485447.3512183

Cited by 34 publications

(13 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Baselines We consider a number of node representation learning baselines including very recent SOTA GCL methods. Baselines trained without labels: DGI (Velickovic et al 2019), GRACE (Zhu et al 2020), MVGRL (Hassani and Khasahmadi 2020), BGRL (Thakoor et al 2021), GCA , COLES (Zhu, Sun, and Koniusz 2021), CCA-SSG (Zhang et al 2021), Ariel (Feng et al 2022a) and SimGRACE (Xia et al 2022). Baselines trained with labels: GCN (Kipf and Welling 2016), GAT (Velickovic et al 2017) and InfoGCL (Xu et al 2021).…”

Section: Methodsmentioning

confidence: 99%

“…We can see that MA-GCL can achieve SOTA performance on 5 out of 6 graph benchmarks, and the relative improvement can go up to 2.7%. Considering that the public splits on Cora, Citeseer and PubMed might not be representative, we also investigate another benchmark setting (Feng et al 2022a) with random splits on these three datasets, and compare with the most competitive baselines. As shown in Table 2, MA-GCL consistently outperforms baseline methods, which demonstrates the effectiveness of MA-GCL.…”

Section: Comparison With Baseline Methodsmentioning

confidence: 99%

“…BGRL (Thakoor et al 2021) performs GCL without negative pairs inspired by BYOL (Grill et al 2020). ARIEL (Feng et al 2022a) further introduces an extra adversarial view to GCL. Recent methods propose to generate views in adaptive ways.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

MA-GCL: Model Augmentation Tricks for Graph Contrastive Learning

Gong

Yang

Shi

2023

AAAI

View full text Add to dashboard Cite

Contrastive learning (CL), which can extract the information shared between different contrastive views, has become a popular paradigm for vision representation learning. Inspired by the success in computer vision, recent work introduces CL into graph modeling, dubbed as graph contrastive learning (GCL). However, generating contrastive views in graphs is more challenging than that in images, since we have little prior knowledge on how to significantly augment a graph without changing its labels. We argue that typical data augmentation techniques (e.g., edge dropping) in GCL cannot generate diverse enough contrastive views to filter out noises. Moreover, previous GCL methods employ two view encoders with exactly the same neural architecture and tied parameters, which further harms the diversity of augmented views. To address this limitation, we propose a novel paradigm named model augmented GCL (MA-GCL), which will focus on manipulating the architectures of view encoders instead of perturbing graph inputs. Specifically, we present three easy-to-implement model augmentation tricks for GCL, namely asymmetric, random and shuffling, which can respectively help alleviate high-frequency noises, enrich training instances and bring safer augmentations. All three tricks are compatible with typical data augmentations. Experimental results show that MA-GCL can achieve state-of-the-art performance on node classification benchmarks by applying the three tricks on a simple base model. Extensive studies also validate our motivation and the effectiveness of each trick. (Code, data and appendix are available at https://github.com/GXM1141/MA-GCL. )

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Comparison With Baseline Methodsmentioning

confidence: 99%

See 1 more Smart Citation

MA-GCL: Model Augmentation Tricks for Graph Contrastive Learning

Gong

Yang

Shi

2023

AAAI

View full text Add to dashboard Cite

show abstract

“…where Ã = A + I n is the adjacency matrix with selfconnections added and I n is the identity matrix, D is the diagonal degree matrix of Ã with D[i, i] = j Ã[i, j]. The two-layer GCN is given as f (A, X) = σ( Âσ( ÂXW (1) )W (2) ),…”

Section: Graph Encodermentioning

confidence: 99%

“…where W (1) and W (2) are the weights of the first and second layer respectively, σ(•) is the activation function.…”

Section: Graph Encodermentioning

confidence: 99%

ARIEL: Adversarial Graph Contrastive Learning

Feng¹,

Jing²,

Zhu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Contrastive learning is an effective unsupervised method in graph representation learning, and the key component of contrastive learning lies in the construction of positive and negative samples. Previous methods usually utilize the proximity of nodes in the graph as the principle. Recently, the data augmentation based contrastive learning method has advanced to show great power in the visual domain, and some works extended this method from images to graphs. However, unlike the data augmentation on images, the data augmentation on graphs is far less intuitive and much harder to provide high-quality contrastive samples, which leaves much space for improvement. In this work, by introducing an adversarial graph view for data augmentation, we propose a simple but effective method, Adversarial Graph Contrastive Learning (ARIEL), to extract informative contrastive samples within reasonable constraints. We develop a new technique called information regularization for stable training and use subgraph sampling for scalability. We generalize our method from node-level contrastive learning to the graph-level by treating each graph instance as a supernode. ARIEL consistently outperforms the current graph contrastive learning methods for both node-level and graph-level classification tasks on real-world datasets. We further demonstrate that ARIEL is more robust in face of adversarial attacks.

show abstract

MDGCL: Graph Contrastive Learning Framework with Multiple Graph Diffusion Methods

Li,

Zhang,

Liu

2024

Neural Process Lett

View full text Add to dashboard Cite

In recent years, some classical graph contrastive learning(GCL) frameworks have been proposed to address the problem of sparse labeling of graph data in the real world. However, in node classification tasks, there are two obvious problems with existing GCL frameworks: first, the stochastic augmentation methods they adopt lose a lot of semantic information; second, the local–local contrasting mode selected by most frameworks ignores the global semantic information of the original graph, which limits the node classification performance of these frameworks. To address the above problems, this paper proposes a novel graph contrastive learning framework, MDGCL, which introduces two graph diffusion methods, Markov and PPR, and a deterministic–stochastic data augmentation strategy while retaining the local–local contrasting mode. Specifically, before using the two stochastic augmentation methods (FeatureDrop and EdgeDrop), MDGCL first uses two deterministic augmentation methods (Markov diffusion and PPR diffusion) to perform data augmentation on the original graph to increase the semantic information, this step ensures subsequent stochastic augmentation methods do not lose too much semantic information. Meanwhile, the diffusion matrices carried by the augmented views contain global semantic information of the original graph, allowing the framework to utilize the global semantic information while retaining the local-local contrasting mode, which further enhances the node classification performance of the framework. We conduct extensive comparative experiments on multiple benchmark datasets, and the results show that MDGCL outperforms the representative baseline frameworks on node classification tasks. Among them, compared with COSTA, MDGCL’s node classification accuracy has been improved by 1.07% and 0.41% respectively on two representative datasets, Amazon-Photo and Coauthor-CS. In addition, we also conduct ablation experiments on two datasets, Cora and CiteSeer, to verify the effectiveness of each improvement work of our framework.

show abstract

Adversarial Graph Contrastive Learning with Information Regularization

Cited by 34 publications

References 16 publications

MA-GCL: Model Augmentation Tricks for Graph Contrastive Learning

MA-GCL: Model Augmentation Tricks for Graph Contrastive Learning

ARIEL: Adversarial Graph Contrastive Learning

MDGCL: Graph Contrastive Learning Framework with Multiple Graph Diffusion Methods

Contact Info

Product

Resources

About