FPGAN: An FPGA Accelerator for Graph Attention Networks With Software and Hardware Co-Optimization

Abstract

The Graph Attention Networks (GATs) exhibit outstanding performance in multiple authoritative node classification benchmark tests (including transductive and inductive). The purpose of this research is to implement an FPGA-based accelerator called FPGAN for graph attention networks that achieves significant improvement on performance and energy efficiency without losing accuracy compared with PyTorch baseline. It eliminates the dependence on digital signal processors (DSPs) and large amounts of on-chip memory and can even work well on low-end FPGA devices. We design FPGAN with software and hardware co-optimization across the full stack from algorithm through architecture. Specifically, we compress model to reduce the model size, quantify features to perform fixed-point calculation, replace multiplication addition cell (MAC) with shift addition units (SAUs) to eliminate the dependence on DSPs, and design an efficient algorithm to approximate SoftMax function. We also adjust the activation functions and fuse operations to further reduce the computation requirement. Moreover, all data is vectorized and aligned for scalable vector computation and efficient memory access. All the above optimizations are integrated into a universal hardware pipeline for various structures of GATs. We evaluate our design on an Inspur F10A board with an Intel Arria 10 GX1150 and 16 GB DDR3 memory. Experimental results show that FPGAN can achieve 7.34 times speedup over Nvidia Tesla V100 and 593 times over Xeon CPU Gold 5115 while maintaining accuracy, and 48 times and 2400 times on energy efficiency respectively.

A survey of field programmable gate array (FPGA)-based graph convolutional neural network accelerators: challenges and opportunities

“…FPGAN ( Yan, Tong & Zhi, 2020 ) is based on FPGA to accelerate the inference process of GAT. FPGAN designs a shift calculation unit for the intensive exp operation in GAT, which eliminates the dependence of computing performance on DSP, and uses an exponential approximation algorithm to fit SoftMax to normalize the attention coefficient.…”

“…To save memory and reduce the computational difficulty, FPGAN ( Yan, Tong & Zhi, 2020 ) compresses the model, and its core idea is to convert the weights to powers of 0 or 2 and judge whether retraining is required by observing the loss of accuracy after conversion. If the compression accuracy loss for one set is within a reasonable range, start the next set of compressions.…”

“…It is currently a popular and effective method to design accelerators for corresponding GCNs models using FPGA that take into account fine-grained computation, high parallelism, and programmability, such as AWB-GCN ( Geng et al, 2020 ), LW-GCN ( Tao et al, 2021 ), FPGAN ( Yan, Tong & Zhi, 2020 ), BoostGCN ( Zhang, Kannan & Prasanna, 2021 ), I-GCN ( Geng et al, 2021b ), etc . These customized FPGA models all completed the inference of GCNs efficiently through specific optimization.…”

A Taxonomy for Classification and Comparison of Dataflows for GNN Accelerators

“…2c, mean pool uses many ThCudaTensor_scatterAddKernels which are also present in the Aggregation phase. Again, similar to previous GNN accelerators [3], [4], [11], [21], [27], [39], [41]- [43], this work will focus only on the Aggregation and Combination phases, as the main kernels in aggregation and combination consume a majority of the GNN inference runtime.…”

“…Computing GNN inference requires a mix of memory and compute intensive operations, which commodity CPUs, GPUs and traditional DNN accelerators do not exploit efficiently [27], [39], [40], [44]. This led to the development of many dedicated GNN accelerators, each with their own design methodology to extract as much performance as possible [3], [4], [11], [21], [27], [39], [41]- [43].…”