Topology-Aware Graph Signal Sampling for Pooling in Graph Neural Networks

Abstract

Evolving networks are complex data structures that emerge in a wide range of systems in science and engineering. Learning expressive representations for such networks that encode their structural connectivity and temporal evolution is essential for downstream data analytics and machine learning applications. In this study, we introduce a self-supervised method for learning representations of temporal networks and employ these representations in the dynamic link prediction task. While temporal networks are typically characterized as a sequence of interactions over the continuous time domain, our study focuses on their discrete-time versions. This enables us to balance the trade-off between computational complexity and precise modeling of the interactions. We propose a recurrent message-passing neural network architecture for modeling the information flow over time-respecting paths of temporal networks. The key feature of our method is the contrastive training objective of the model, which is a combination of three loss functions: link prediction, graph reconstruction, and contrastive predictive coding losses. The contrastive predictive coding objective is implemented using infoNCE losses at both local and global scales of the input graphs. We empirically show that the additional self-supervised losses enhance the training and improve the model's performance in the dynamic link prediction task. The proposed method is tested on Enron, COLAB, and Facebook datasets and exhibits superior results compared to existing models. 1

Graph Structure Prompt Learning: A Novel Methodology to Improve Performance of Graph Neural Networks

“…SAGPool [26] leverages a self-attention graph pooling layer based on hierarchical graph pooling, which can learn hierarchical representation in an end-to-end manner with relatively few parameters. Recent advanced graph neural networks also include EdgePool [27], TopKPool [25], ASAPool [28], MEWISPool [29], GPS [30]. The well-known over-smooth problem prevents the layers of GNNs from going deeper.…”

“…MEWISPool [29]: A graph pooling method based on maximizing mutual information between the pooled graph and the input graph. It employs the Shannon capacity of the graph as an inductive bias during the pooling process.…”

“…Classic and advanced graph neural networks, including ChebNet [13], graph convolution networks (GCN) [14], GraphSage [15], graph attention networks (GAT) [16], LightGCN [17], UniMP [18], ARMA [19], Fused GAT [20], ASDGN [21] etc., were initially designed for node classification tasks. To gather node features for graph classification tasks, various readout functions and pooling mechanisms have been proposed, such as GIN [22], SortPool [23], DiffPool [24], TopKPool [25], SAGPool [26], EdgePool [27], ASAPool [28], and MEWISPool [29], GPS [30], etc. Compared with the thriving models and applications, training methods for improving the performance of the classic graph neural networks are less discussed.…”

Maximum Entropy Weighted Independent Set Pooling for Graph Neural Networks

“…GNNs utilize graph-based pooling techniques to learn hierarchical representations of the input graphs. In a general perspective, one can categorize the graph pooling techniques into the methods which are based on node selection [18,33,34,39], and the methods based on graph coarsening [64,7,52,19,65]. The node-selection-based pooling methods assign an importance score to each node of the graph and select the high score nodes as the pooled nodes.…”

Label Attentive Distillation for GNN-Based Graph Classification

“…Graph Neural Networks (GNNs) (Kipf andWelling 2016, 2017) are adept at converting unstructured data into lowdimensional representations by effectively capturing both node features and topological dependencies. The GNN learning tasks broadly focused on node classification, link prediction, and graph classification (You et al 2021;Wang et al 2021), which have demonstrated their effectiveness in various fields such as protein-to-protein interaction (Nouranizadeh et al 2021), molecular medicine (Li et al 2022b) information retrieval (Chen et al 2022), etc. This paper focuses on graph classification, a graph-level task that aims to learn a graph representation with GNNs to predict the graph labels.…”