GNNAdvisor: An Adaptive and Efficient Runtime System for GNN Acceleration on GPUs

Wang, Yuke; Feng, Boyuan; Li, Gushu; Li, Shuangchen; Deng, Lei; Xie, Yuan; Ding, Yufei

doi:10.48550/arxiv.2006.06608

Cited by 7 publications

(32 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In response to the challenges of GNN computing, several works have surfaced that attempt to improve the performance and efficiency of GNNs either from a software perspective, i.e. adapting the operations to better match the capabilities of CPUs or GPUs [37]- [39]; or from a hardware perspective, i.e. designing custom processors tailored to the demands of GNNs [40]- [43].…”

Section: Study [Reference] (Year) Contributionsmentioning

confidence: 99%

Computing Graph Neural Networks: A Survey from Algorithms to Accelerators

et al. 2021

View full text Add to dashboard Cite

Graph Neural Networks (GNNs) have exploded onto the machine learning scene in recent years owing to their capability to model and learn from graph-structured data. Such an ability has strong implications in a wide variety of fields whose data are inherently relational, for which conventional neural networks do not perform well. Indeed, as recent reviews can attest, research in the area of GNNs has grown rapidly and has lead to the development of a variety of GNN algorithm variants as well as to the exploration of ground-breaking applications in chemistry, neurology, electronics, or communication networks, among others. At the current stage research, however, the efficient processing of GNNs is still an open challenge for several reasons. Besides of their novelty, GNNs are hard to compute due to their dependence on the input graph, their combination of dense and very sparse operations, or the need to scale to huge graphs in some applications. In this context, this article aims to make two main contributions. On the one hand, a review of the field of GNNs is presented from the perspective of computing. This includes a brief tutorial on the GNN fundamentals, an overview of the evolution of the field in the last decade, and a summary of operations carried out in the multiple phases of different GNN algorithm variants. On the other hand, an in-depth analysis of current software and hardware acceleration schemes is provided, from which a hardware-software, graph-aware, and communication-centric vision for GNN accelerators is distilled.

show abstract

Section: Study [Reference] (Year) Contributionsmentioning

confidence: 99%

Computing Graph Neural Networks: A Survey from Algorithms to Accelerators

et al. 2021

View full text Add to dashboard Cite

show abstract

“…The acceleration of GNN workloads is an active area of research that distinguishes between software and hardware acceleration [2]. On the one hand, software acceleration for GNNs aims at exploiting the knowledge of the graph properties to better adapt the workload to the underlying hardware [4], [10], [14], [16], [20], [28], [34]- [37], [45]. This includes techniques such as intelligent partitioning [34], sparsity-aware workload management [37], vertex reordering [4], or the caching of partial aggregations to avoid redundant sums [16].…”

Section: Related Workmentioning

confidence: 99%

“…On the one hand, software acceleration for GNNs aims at exploiting the knowledge of the graph properties to better adapt the workload to the underlying hardware [4], [10], [14], [16], [20], [28], [34]- [37], [45]. This includes techniques such as intelligent partitioning [34], sparsity-aware workload management [37], vertex reordering [4], or the caching of partial aggregations to avoid redundant sums [16]. These techniques are either specific for GPUs, such as the dataflow constructs in Neugraph [28], or orthogonal to the dataflow approach.…”

Section: Related Workmentioning

confidence: 99%

A Taxonomy for Classification and Comparison of Dataflows for GNN Accelerators

Garg¹,

Qin²,

Muñoz-Martínez³

et al. 2021

View full text Add to dashboard Cite

Recently, Graph Neural Networks (GNNs) have received a lot of interest because of their success in learning representations from graph structured data. However, GNNs exhibit different compute and memory characteristics compared to traditional Deep Neural Networks (DNNs). Graph convolutions require feature aggregations from neighboring nodes (known as the aggregation phase), which leads to highly irregular data accesses. GNNs also have a very regular compute phase that can be broken down to matrix multiplications (known as the combination phase). All recently proposed GNN accelerators utilize different dataflows and microarchitecture optimizations for these two phases. Different communication strategies between the two phases have been also used. However, as more custom GNN accelerators are proposed, the harder it is to qualitatively classify them and quantitatively contrast them. In this work, we present a taxonomy to describe several diverse dataflows for running GNN inference on accelerators. This provides a structured way to describe and compare the design-space of GNN accelerators.

show abstract

“…There exist several recent frameworks for computations on GNNs [35,42,44,45,51,56,80,85,89,92,98]. While we use Pytorch Geometric, motif prediction can be integrated into any of these frameworks to enhance their processing capabilities.…”

Section: Related Workmentioning

confidence: 99%

Motif Prediction with Graph Neural Networks

Besta¹,

Grob²,

Miglioli³

et al. 2021

Preprint

View full text Add to dashboard Cite

Link prediction is one of the central problems in graph mining. However, recent studies highlight the importance of higher-order network analysis, where complex structures called motifs are the first-class citizens. We first show that existing link prediction schemes fail to effectively predict motifs. To alleviate this, we establish a general motif prediction problem and we propose several heuristics that assess the chances for a specified motif to appear. To make the scores realistic, our heuristics consider -among others -correlations between links, i.e., the potential impact of some arriving links on the appearance of other links in a given motif. Finally, for highest accuracy, we develop a graph neural network (GNN) architecture for motif prediction. Our architecture offers vertex features and sampling schemes that capture the rich structural properties of motifs. While our heuristics are fast and do not need any training, GNNs ensure highest accuracy of predicting motifs, both for dense (e.g., k-cliques) and for sparse ones (e.g., k-stars). We consistently outperform the best available competitor by more than 10% on average and up to 32% in area under the curve. Importantly, the advantages of our approach over schemes based on uncorrelated link prediction increase with the increasing motif size and complexity. We also successfully apply our architecture for predicting more arbitrary clusters and communities, illustrating its potential for graph mining beyond motif analysis.

show abstract

GNNAdvisor: An Adaptive and Efficient Runtime System for GNN Acceleration on GPUs

Cited by 7 publications

References 37 publications

Computing Graph Neural Networks: A Survey from Algorithms to Accelerators

Computing Graph Neural Networks: A Survey from Algorithms to Accelerators

A Taxonomy for Classification and Comparison of Dataflows for GNN Accelerators

Motif Prediction with Graph Neural Networks

Contact Info

Product

Resources

About