Hypergraph Pre-training with Graph Neural Networks

Abstract

Despite the prevalence of hypergraphs in a variety of high-impact applications, there are relatively few works on hypergraph representation learning, most of which primarily focus on hyperlink prediction, and often restricted to the transductive learning setting. Among others, a major hurdle for effective hypergraph representation learning lies in the label scarcity of nodes and/or hyperedges. To address this issue, this paper presents an end-to-end, bi-level pre-training strategy with Graph Neural Networks for hypergraphs. The proposed framework named HyperGene bears three distinctive advantages. First, it is capable of ingesting the labeling information when available, but more importantly, it is mainly designed in the self-supervised fashion which significantly broadens its applicability. Second, at the heart of the proposed HyperGene are two carefully designed pretexts, one on the node level and the other on the hyperedge level, which enable us to encode both the local and the global context in a mutually complementary way. Third, the proposed framework can work in both transductive and inductive settings. When applying the two proposed pretexts in tandem, it can accelerate the adaptation of the knowledge from the pre-trained model to downstream applications in the transductive setting, thanks to the bi-level nature of the proposed method. The extensive experimental results demonstrate that: (1) HyperGene achieves up to 5.69% improvements in hyperedge classification, and (2) improves pre-training efficiency by up to 42.80% on average.

PhyGCN: Pre-trained Hypergraph Convolutional Neural Networks with Self-supervised Learning

“…While self-supervised training has been extensively studied for graphs with pairwise links [16][17][18][19][20][21][22][23], few works have explored it for hypergraph representation learning [27]. Our focus is on leveraging the attention mechanism [28] to predict hyperedges with arbitrary lengths and capture the indecomposiblity of a hyperedge, allowing PhyGCN to extract useful information from complex hypergraph structures.…”

“…While SSL methods have been proposed for GNNs [16][17][18][19][20][21][22][23], their application to hypergraph learning is underexplored. Noteworthy works [24][25][26] have targeted specific applications such as recommender systems, while others [27] propose pre-training GNNs on hypergraphs, primarily focusing on hyperedge prediction. A detailed discussion of related works can be found in Supplementary Note A.…”

“…Simple average pooling of node embeddings has been found insufficient to model a hyperedge [29]. Although some works [24][25][26][27] have explored the potential of self-supervised learning on hypergraphs, our PhyGCN method is the first straightforward and effective approach to utilize hypergraph structure and can be generally applied to many tasks.…”

Multimodal Self-Supervised Learning Networks for In-vehicle Noise Prediction

“…Self-supervised hypergraph learning. Recently, there have been a handful of works to study self-supervised learning on hypergraphs [14]- [16], [40], [42]. HyperGene [42] adopts bi-level (node-and hyperedge-level) self-supervised tasks to effectively learn group-wise relations.…”

“…Recently, there have been a handful of works to study self-supervised learning on hypergraphs [14]- [16], [40], [42]. HyperGene [42] adopts bi-level (node-and hyperedge-level) self-supervised tasks to effectively learn group-wise relations. However, it adopts a clique expansion to transform a hypergraph into a simple graph, which incurs a significant loss of high-order information and does not employ contrastive learning.…”