Neural Graph Matching for Pre-training Graph Neural Networks

Hou, Yupeng; Hu, Binbin; Zhao, Wayne Xin; Zhang, Zhiqiang; Zhou, Jun; Wen, Ji-Rong

doi:10.1137/1.9781611977172.20

Cited by 141 publications

(356 citation statements)

References 0 publications

Supporting

Mentioning

354

Contrasting

Order By: Relevance

“…In the first method, edge weights are added to the features of the nodes (Hu et al, 2020a;Li et al, 2020). In the second approach, edge weights are treated the same way as node features in the Aggregation function of the GNNs (Gilmer et al, 2017;Xu et al, 2019;Kipf & Welling, 2017;Hu et al, 2020b).…”

Section: Discussionmentioning

confidence: 99%

SoftEdge: Regularizing Graph Classification with Random Soft Edges

Guo¹,

Sun²

2022

Preprint

View full text Add to dashboard Cite

Graph data augmentation plays a vital role in regularizing Graph Neural Networks (GNNs), which leverage information exchange along edges in graphs, in the form of message passing, for learning. Due to their effectiveness, simple edge and node manipulations (e.g., addition and deletion) have been widely used in graph augmentation. In this paper, we identify a limitation in such a common augmentation technique. That is, simple edge and node manipulations can create graphs with an identical structure or indistinguishable structures to message passing GNNs but of conflict labels, leading to the sample collision issue and thus the degradation of model performance.To address this problem, we propose SoftEdge, which assigns random weights to a portion of the edges of a given graph to construct dynamic neighborhoods over the graph. We prove that SoftEdge creates collision-free augmented graphs. We also show that this simple method obtains superior accuracy to popular node and edge manipulation approaches and notable resilience to the accuracy degradation with the GNN depth.

show abstract

Section: Discussionmentioning

confidence: 99%

SoftEdge: Regularizing Graph Classification with Random Soft Edges

Guo¹,

Sun²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…• Mutagenicity [50], [51] has 4, 377 molecule graphs, where each graph is labeled with one of two labels: mutagenic and non-mutagenic, based on the mutagenic effect on a bacterium. We trained a GIN model [23], [52] to perform the binary classification. • REDDIT-MULTI-5K [53] has 4, 999 social networks labeled with five different classes to indicate the topics of question/answer communities.…”

Section: Dataset Descriptionmentioning

confidence: 99%

Reinforced Causal Explainer for Graph Neural Networks

Wang

Zhang

et al. 2023

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Explainability is crucial for probing graph neural networks (GNNs), answering questions like "Why the GNN model makes a certain prediction?". Feature attribution is a prevalent technique of highlighting the explanatory subgraph in the input graph, which plausibly leads the GNN model to make its prediction. Various attribution methods have been proposed to exploit gradient-like or attention scores as the attributions of edges, then select the salient edges with top attribution scores as the explanation. However, most of these works make an untenable assumption -the selected edges are linearly independent -thus leaving the dependencies among edges largely unexplored, especially their coalition effect. We demonstrate unambiguous drawbacks of this assumptionmaking the explanatory subgraph unfaithful and verbose. To address this challenge, we propose a reinforcement learning agent, Reinforced Causal Explainer (RC-Explainer). It frames the explanation task as a sequential decision process -an explanatory subgraph is successively constructed by adding a salient edge to connect the previously selected subgraph. Technically, its policy network predicts the action of edge addition, and gets a reward that quantifies the action's causal effect on the prediction. Such reward accounts for the dependency of the newly-added edge and the previously-added edges, thus reflecting whether they collaborate together and form a coalition to pursue better explanations. It is trained via policy gradient to optimize the reward stream of edge sequences. As such, RC-Explainer is able to generate faithful and concise explanations, and has a better generalization power to unseen graphs. When explaining different GNNs on three graph classification datasets, RC-Explainer achieves better or comparable performance to state-of-the-art approaches w.r.t. two quantitative metrics: predictive accuracy, contrastivity, and safely passes sanity checks and visual inspections. Codes and datasets are available at https://github.com/xiangwang1223/reinforced causal explainer.

show abstract

“…However, if there are multi-dimensional edge features, the SpMM/GEMM formulation no longer holds. For example, GIN [20] with edge embeddings is formulated as below:…”

Section: Limitationsmentioning

confidence: 99%

“…All of these models also use global average pooling, and an output head with a single linear layer. For PNA, we use 4 layers with an node embedding dimension of 80, global average pooling, and an MLP-ReLU head with sizes (40,20,1). For DGN, we use 4 layers and a node embedding dimension of 100, global average pooling, and an MLP-ReLU head with sizes (50, 25, 1).…”

Section: Model and Implementation Detailsmentioning

confidence: 99%

FlowGNN: A Dataflow Architecture for Universal Graph Neural Network Inference via Multi-Queue Streaming

Sarkar¹,

Stefan²,

He³

et al. 2022

Preprint

View full text Add to dashboard Cite

Graph neural networks (GNNs) have recently exploded in popularity thanks to their broad applicability to graph-related problems such as quantum chemistry, drug discovery, and high energy physics. However, meeting demand for novel GNN models and fast inference simultaneously is challenging because of the gap between developing efficient accelerators and the rapid creation of new GNN models. Prior art focuses on the acceleration of specific classes of GNNs, such as Graph Convolutional Network (GCN), but lacks the generality to support a wide range of existing or new GNN models. Meanwhile, most work rely on graph pre-processing to exploit data locality, making them unsuitable for real-time applications. To address these limitations, in this work, we propose a generic dataflow architecture for GNN acceleration, named FlowGNN, which can flexibly support the majority of message-passing GNNs. The contributions are three-fold. First, we propose a novel and scalable dataflow architecture, which flexibly supports a wide range of GNN models with message-passing mechanism. The architecture features a configurable dataflow optimized for simultaneous computation of node embedding, edge embedding, and message passing, which is generally applicable to all models. We also propose a rich library of model-specific components. Second, we deliver ultra-fast real-time GNN inference without any graph pre-processing, making it agnostic to dynamically changing graph structures. Third, we verify our architecture on the Xilinx Alveo U50 FPGA board and measure the on-board end-to-end performance. We achieve a speed-up of up to 51-254× against CPU (6226R) and 1.3-477× against GPU (A6000) (with batch sizes 1 through 1024); we also outperform the SOTA GNN accelerator I-GCN by 1.03× and 1.25× across two datasets. Our implementation code and on-board measurement are publicly available on GitHub. 1

show abstract

Neural Graph Matching for Pre-training Graph Neural Networks

Cited by 141 publications

References 0 publications

SoftEdge: Regularizing Graph Classification with Random Soft Edges

SoftEdge: Regularizing Graph Classification with Random Soft Edges

Reinforced Causal Explainer for Graph Neural Networks

FlowGNN: A Dataflow Architecture for Universal Graph Neural Network Inference via Multi-Queue Streaming

Contact Info

Product

Resources

About