Simple Unsupervised Graph Representation Learning

Abstract

In this paper, we propose a simple unsupervised graph representation learning method to conduct effective and efficient contrastive learning. Specifically, the proposed multiplet loss explores the complementary information between the structural information and neighbor information to enlarge the inter-class variation, as well as adds an upper bound loss to achieve the finite distance between positive embeddings and anchor embeddings for reducing the intra-class variation. As a result, both enlarging inter-class variation and reducing intra-class variation result in small generalization error, thereby obtaining an effective model. Furthermore, our method removes widely used data augmentation and discriminator from previous graph contrastive learning methods, meanwhile available to output low-dimensional embeddings, leading to an efficient model. Experimental results on various real-world datasets demonstrate the effectiveness and efficiency of our method, compared to state-of-the-art methods. The source codes are released at https://github.com/YujieMo/SUGRL.

Graph Soft-Contrastive Learning via Neighborhood Ranking

“…Binary COSTA (Zhang et al 2022) HFA: adding random noise into node embeddings. Binary SUGRL (Mo et al 2022) Nonsynthetic: 1-hop neighbours and the output of 2 different models. Binary AFGRL (Lee, Lee, and Park 2022) Nonsynthetic: 1-hop neighbours and KNN similar nodes.…”

“…Fine-Grained Contrastive Justification Synthetic-Based Nonsynthetic-Based GRACE (Zhu et al 2020) COSTA (Zhang et al 2022) SUGRL (Mo et al 2022) AFGRL (Lee, Lee, and Park 2022) OURS our proposed GSCL. Current GCL adopts the binary contrastive justification setting, that aims to make similarities between samples in the positive views as larger as possible than samples in the negative views.…”

“…We classify this as hiddenfeature/model augmentation. Furthermore, SUGRL feeds the same graph into a multi-layer perceptron and a GCN to generate graph views (Mo et al 2022); AFGRL generates graph views by searching nodes that can serve as positive samples via KNN search in the representation space. (Lee, Lee, and Park 2022).…”

“…On the contrary, nonsynthetic-based GCL methods produce different views of representations without generating any artificial components. For example, SUGRL feeds the same graph into different embedding models to generate the positive samples and aims to maximize the agreement between them (Mo et al 2022); AF-GRL searches similar samples in the neighborhood through K-Nearest Neighbor (KNN) algorithm to construct the positive view (Lee, Lee, and Park 2022).…”

Building Shortcuts between Distant Nodes with Biaffine Mapping for Graph Convolutional Networks

“…Most of these models achieve best performance when using two or three convolution layers and show dramatic performance degradation for more layers. A more model SUGRL [28] leverages feature shuffling and graph sampling to explore the complementary information between structural information and neighborhood information to expand interclass variation in unsupervised setting. MixHop [11] is one successful attempt to receive the information of multi-level neighbors with shallow architecture.…”