FlowGNN: A Dataflow Architecture for Universal Graph Neural Network Inference via Multi-Queue Streaming

Sarkar, Rishov; Stefan, Abi-Karam,; He, Yuan; Sathidevi, Lakshmi; Hao, Cong

doi:10.48550/arxiv.2204.13103

Cited by 3 publications

(3 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Results show their designs achieve millisecond level latency. They also propose FlowGNN [43] which can flexibly support the majority of message-passing GNNs. [11] proposes a GCN accelerator named GROW with Gustavson's algorithm to architect a sparse-dense GEMM accelerator with row-wise product.…”

Section: Related Workmentioning

confidence: 99%

LL-GNN: Low Latency Graph Neural Networks on FPGAs for Particle Detectors

Que¹,

Loo²,

Fan³

et al. 2022

Preprint

View full text Add to dashboard Cite

This work proposes a novel reconfigurable architecture for low latency Graph Neural Network (GNN) design specifically for particle detectors. Accelerating GNNs for particle detectors is challenging since it requires sub-microsecond latency to deploy the networks for online event selection in the Level-1 triggers at the CERN Large Hadron Collider experiments. This paper proposes a custom code transformation with strength reduction for the matrix multiplication operations in the interaction-network based GNNs with fully connected graphs, which avoids the costly multiplication of the adjacency matrix with the input feature matrix. It exploits sparsity patterns as well as binary adjacency matrices, and avoids irregular memory access, leading to a reduction in latency and improvement in hardware efficiency. In addition, we introduce an outer-product based matrix multiplication approach which is enhanced by the strength reduction for low latency design. Also, a fusion step is introduced to further reduce the design latency. Furthermore, an GNN-specific algorithm-hardware co-design approach is presented which not only finds a design with a much better latency but also finds a high accuracy design under a given latency constraint. Finally, a customizable template for this low latency GNN hardware architecture has been designed and open-sourced, which enables the generation of low-latency FPGA designs with efficient resource utilization using a high-level synthesis tool. Evaluation results show that our FPGA implementation is up to 24 times faster and consumes up to 45 times less power than a GPU implementation. Compared to our previous FPGA implementations, this work achieves 6.51 to 16.7 times lower latency. Moreover, the latency of our FPGA design is sufficiently low to enable deployment of GNNs in a sub-microsecond, real-time collider trigger system, enabling it to benefit from improved accuracy.

show abstract

Section: Related Workmentioning

confidence: 99%

LL-GNN: Low Latency Graph Neural Networks on FPGAs for Particle Detectors

Que¹,

Loo²,

Fan³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…FlowGNN ( Sarkar et al, 2022 ) proposes a general-purpose GNN acceleration framework using high-level synthesis to deal with the imbalanced development between new GNN algorithms and new accelerators. Unlike previous class-specific GNN model accelerators, FlowGNN supports edge embeddings for widely popular GNN models and can be extended to new models.…”

Section: Fpga Based Hardware Acceleratorsmentioning

confidence: 99%

A survey of field programmable gate array (FPGA)-based graph convolutional neural network accelerators: challenges and opportunities

Yao

Tang

et al. 2022

PeerJ Computer Science

View full text Add to dashboard Cite

Graph convolutional networks (GCNs) based on convolutional operations have been developed recently to extract high-level representations from graph data. They have shown advantages in many critical applications, such as recommendation system, natural language processing, and prediction of chemical reactivity. The problem for the GCN is that its target applications generally pose stringent constraints on latency and energy efficiency. Several studies have demonstrated that field programmable gate array (FPGA)-based GCNs accelerators, which balance high performance and low power consumption, can continue to achieve orders-of-magnitude improvements in the inference of GCNs models. However, there still are many challenges in customizing FPGA-based accelerators for GCNs. It is necessary to sort out the current solutions to these challenges for further research. For this purpose, we first summarize the four challenges in FPGA-based GCNs accelerators. Then we introduce the process of the typical GNN algorithm and several examples of representative GCNs. Next, we review the FPGA-based GCNs accelerators in recent years and introduce their design details according to different challenges. Moreover, we compare the key metrics of these accelerators, including resource utilization, performance, and power consumption. Finally, we anticipate the future challenges and directions for FPGA-based GCNs accelerators: algorithm and hardware co-design, efficient task scheduling, higher generality, and faster development.

show abstract

“…Low-latency GNN inference is needed in many real-world applications, such as traffic prediction [1], scientific simulation [2], etc. While many techniques [3], [4], [5], [6], [7], [8], [9] have been proposed to accelerate GNN inference, no work has systematically studied the data sparsity in GNNs to reduce the inference latency. GNNs ( [10], [11]) involve various computation kernels, where there are three types of data sparsity:…”

Section: Introductionmentioning

confidence: 99%

Dynasparse: Accelerating GNN Inference through Dynamic Sparsity Exploitation

Zhang¹,

Prasanna²

2023

Preprint

View full text Add to dashboard Cite

Graph Neural Network (GNN) inference is used in many real-world applications. Data sparsity in GNN inference, including sparsity in the input graph and the GNN model, offer opportunities to further speed up inference. Also, many pruning techniques have been proposed for model compression that increase the data sparsity of GNNs.We propose Dynasparse, a comprehensive hardware-software codesign on FPGA to accelerate GNN inference through dynamic sparsity exploitation. For this, we decouple the GNN computation kernels from the basic computation primitives, and explore hardware-software codesign as follows: 1) Hardware design: We propose a novel unified accelerator design on FPGA to efficiently execute various computation primitives. We develop a customized soft processor that is tightly coupled with the accelerator to execute a runtime system. Moreover, we develop efficient hardware mechanisms to profile the data sparsity and perform on-thefly data format transformation to prepare the input data for various computation primitives; 2) Software design: We develop a runtime system that works synergistically with the accelerator to perform dynamic kernel-to-primitive mapping based on data sparsity. We implement Dynasparse on a state-of-the-art FPGA platform, Xilinx Alveo U250, and evaluate the design using widely used GNN models (GCN, GraphSAGE, GIN and SGC). For the above GNN models and various input graphs, the proposed accelerator and dynamic kernel-to-primitive mapping reduces the inference latency by 3.73× on the average compared with the static mapping strategies employed in the state-ofthe-art GNN accelerators. Compared with state-of-the-art CPU (GPU) implementations, Dynasparse achieves up to 56.9× (2.37×) speedup in end-to-end latency. Compared with state-of-the-art FPGA implementations, Dynasparse achieves 2.7× speedup in accelerator execution latency.Index Terms-Graph neural network, hardware-software codesign, hardware architecture, runtime system• We develop a complete system on FPGA with the following innovations in hardware design:a novel hardware architecture, named Agile Computation Module, consisting of multiple Computation

show abstract

FlowGNN: A Dataflow Architecture for Universal Graph Neural Network Inference via Multi-Queue Streaming

Cited by 3 publications

References 39 publications

LL-GNN: Low Latency Graph Neural Networks on FPGAs for Particle Detectors

LL-GNN: Low Latency Graph Neural Networks on FPGAs for Particle Detectors

A survey of field programmable gate array (FPGA)-based graph convolutional neural network accelerators: challenges and opportunities

Dynasparse: Accelerating GNN Inference through Dynamic Sparsity Exploitation

Contact Info

Product

Resources

About