Self-Supervised Learning on Graphs: Contrastive, Generative, or Predictive

Wu, Lirong; Lin, Haitao; Tan, Cheng; Gao, Zhangyang; Li, Stan Z.

doi:10.1109/tkde.2021.3131584

Cited by 139 publications

(67 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Considering the advantages of contrastive learning (see Section 3.2), GNN-based contrastive learning approaches are suitable for extracting the embedding vectors of graphs of SVG elements in our structure-aware approach. Existing GNN-based contrastive learning approaches are mainly applied to three types of tasks [53], including node-level tasks such as node classification (e.g., DGI [45]), edge-level tasks such as link prediction (e.g., BiGi [3]) and graph-level tasks such as graph classification (e.g., InfoGraph [42]). Since we aim to represent the structural information in the graph of SVG elements with embedding vectors, InfoGraph [42], one of the state-of-the-art methods for graph embedding, is applied in our approach.…”

Section: Graph Neural Networkmentioning

confidence: 99%

Structure-aware Visualization Retrieval

Li,

Wang,

et al. 2022

Preprint

View full text Add to dashboard Cite

With the wide usage of data visualizations, a huge number of Scalable Vector Graphic (SVG)-based visualizations have been created and shared online. Accordingly, there has been an increasing interest in exploring how to retrieve perceptually similar visualizations from a large corpus, since it can benefit various downstream applications such as visualization recommendation. Existing methods mainly focus on the visual appearance of visualizations by regarding them as bitmap images. However, the structural information intrinsically existing in SVG-based visualizations is ignored. Such structural information can delineate the spatial and hierarchical relationship among visual elements, and characterize visualizations thoroughly from a new perspective. This paper presents a structureaware method to advance the performance of visualization retrieval by collectively considering both the visual and structural information. We extensively evaluated our approach through quantitative comparisons, a user study and case studies. The results demonstrate the effectiveness of our approach and its advantages over existing methods.

show abstract

Section: Graph Neural Networkmentioning

confidence: 99%

Structure-aware Visualization Retrieval

Li,

Wang,

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…For example, GRACE [17] contrasts node-node pairs, GraphCL [16] considers graph-graph pairs, while DGI [13], InfoGraph [19], and MVGRL [14] constructs graph-node contrasting pairs. Although there has been several survey papers on self-supervised graph representation learning [20][21][22], to the best of our knowledge, none of existing work provides rigorous empirical evidence on the impact of each component in GCL. In this work, we proposes a rather complete dissection of existing work and provides empirical insights into building an effective GCL algorithm.…”

Section: Background and Related Workmentioning

confidence: 99%

“…We note that although there has been several survey papers on self-supervised graph representation learning [20][21][22], to the best of our knowledge, none of existing work provides rigorous empirical evidence on the impact of each component in GCL.…”

Section: Introductionmentioning

confidence: 99%

An Empirical Study of Graph Contrastive Learning

Zhu¹,

Xu²,

Liu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Graph Contrastive Learning (GCL) establishes a new paradigm for learning graph representations without human annotations. Although remarkable progress has been witnessed recently, the success behind GCL is still left somewhat mysterious. In this work, we first identify several critical design considerations within a general GCL paradigm, including augmentation functions, contrasting modes, contrastive objectives, and negative mining techniques. Then, to understand the interplay of different GCL components, we conduct extensive, controlled experiments over a set of benchmark tasks on datasets across various domains. Our empirical studies suggest a set of general receipts for effective GCL, e.g., simple topology augmentations that produce sparse graph views bring promising performance improvements; contrasting modes should be aligned with the granularities of end tasks. In addition, to foster future research and ease the implementation of GCL algorithms, we develop an easy-to-use library PyGCL, featuring modularized CL components, standardized evaluation, and experiment management. We envision this work to provide useful empirical evidence of effective GCL algorithms and offer several insights for future research.

show abstract

“…GNNs models couple with contrastive learning to learn graph or node level representations without relying on supervisory data [31]. Then the trained model can transfer the learned representations to a priori unknown downstream tasks.…”

Section: B Contrastive Learningmentioning

confidence: 99%

Pre-training Graph Neural Network for Cross Domain Recommendation

Wang¹,

Liang²,

Liu³

et al. 2021

Preprint

View full text Add to dashboard Cite

A recommender system predicts users' potential interests in items, where the core is to learn user/item embeddings. Nevertheless, it suffers from the data-sparsity issue, which the cross-domain recommendation can alleviate. However, most prior works either jointly learn the source domain and target domain models, or require side-features. However, jointly training and side features would affect the prediction on the target domain as the learned embedding is dominated by the source domain containing bias information. Inspired by the contemporary arts in pre-training from graph representation learning, we propose a pre-training and fine-tuning diagram for cross-domain recommendation. We devise a novel Pre-training Graph Neural Network for Cross-Domain Recommendation (PCRec), which adopts the contrastive self-supervised pre-training of a graph encoder. Then, we transfer the pre-trained graph encoder to initialize the node embeddings on the target domain, which benefits the fine-tuning of the single domain recommender system on the target domain. The experimental results demonstrate the superiority of PCRec. Detailed analyses verify the superiority of PCRec in transferring information while avoiding biases from source domains.

show abstract

Self-Supervised Learning on Graphs: Contrastive, Generative, or Predictive

Cited by 139 publications

References 40 publications

Structure-aware Visualization Retrieval

Structure-aware Visualization Retrieval

An Empirical Study of Graph Contrastive Learning

Pre-training Graph Neural Network for Cross Domain Recommendation

Contact Info

Product

Resources

About