DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphs

Zheng, Da; Ma, Chao; Wang, Minjie; Zhou, Jinjing; Su, Qidong; Song, Xiang; Gan, Quan; Zhang, Zheng; Karypis, George

doi:10.1109/ia351965.2020.00011

Cited by 138 publications

(92 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, GraphSAINT [Zeng et al, 2020] utilizes three samplers, i.e., node sampler, edge sampler, and random walk sampler, to sample nodes (or edges) and construct a subgraph in each batch. Moreover, subgraph sampling can be parallelized at the processing unit level with a training scheduler aided [Zeng et al, 2019], making the training efficient and easily scalable. Such parallelization takes good advantage of the property that subgraphs can be sampled independently.…”

Section: Graph-level Improvmentsmentioning

confidence: 99%

“…SGCN formulates graph sparsification as an optimization problem and resolves it via an alternating direction method of multipliers (ADMM) approach[Boyd et al, 2011]. Additionally, GAUG[Zhao et al, 2021] proposes two variants: GAUG-M utilizes an edge predictor to acquire probabilities of edges in a graph, and modifies input graphs based on the predicted probabilities; GAUG-O integrates the edge predictor and GNN model to jointly promote edge prediction and model accuracy.…”

mentioning

confidence: 99%

“…Generally, the cost of aggregating neighbor representation in a full-batch manner largely depends on the number of edges. sampling methods, e.g., AS-GCN, GraphSAINT[Zeng et al, 2020], can be leveraged to eliminate variance that introduced by probabilistic sampling. By regarding redundant edges in a graph as noise, sparsification methods such as NeuralSparse[Zheng et al, 2020a] can learn a specific strategy to remove task-irrelevant edges, achieving an effect of denoising.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Survey on Graph Neural Network Acceleration: An Algorithmic Perspective

Liu¹,

Yan²,

Deng³

et al. 2022

Preprint

View full text Add to dashboard Cite

Graph neural networks (GNNs) have been a hot spot of recent research and are widely utilized in diverse applications. However, with the use of huger data and deeper models, an urgent demand is unsurprisingly made to accelerate GNNs for more efficient execution. In this paper, we provide a comprehensive survey on acceleration methods for GNNs from an algorithmic perspective. We first present a new taxonomy to classify existing acceleration methods into five categories. Based on the classification, we systematically discuss these methods and highlight their correlations. Next, we provide comparisons from aspects of the efficiency and characteristics of these methods. Finally, we suggest some promising prospects for future research.

show abstract

Section: Graph-level Improvmentsmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

Survey on Graph Neural Network Acceleration: An Algorithmic Perspective

Liu¹,

Yan²,

Deng³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…There are also a lot of works that use supervised learning and graph neural network (GNN) libraries [13,22,30,31,37,45,46,50,52] to solve graph related applications. Lately, a few researchers start to employ supervised GNN learning to solve combinatorial optimization problems.…”

Section: Related Workmentioning

confidence: 99%

OpenGraphGym: A Parallel Reinforcement Learning Framework for Graph Optimization Problems

Zheng

Wang

Song

2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

This paper presents an open-source, parallel AI environment (named OpenGraphGym) to facilitate the application of reinforcement learning (RL) algorithms to address combinatorial graph optimization problems. This environment incorporates a basic deep reinforcement learning method, and several graph embeddings to capture graph features, it also allows users to rapidly plug in and test new RL algorithms and graph embeddings for graph optimization problems. This new opensource RL framework is targeted at achieving both high performance and high quality of the computed graph solutions. This RL framework forms the foundation of several ongoing research directions, including 1) benchmark works on different RL algorithms and embedding methods for classic graph problems; 2) advanced parallel strategies for extreme-scale graph computations, as well as 3) performance evaluation on real-world graph solutions.

show abstract

“…For instance, sampling-based approaches aim at reducing the neighborhood size via layer sampling [2], [3], [9], clustering based sampling [5] and graph sampling [27] techniques; these prior works approach this problem purely from an algorithmic angle. A few recent works [23], [30] investigate the topic of distributed multi-GPU training of GNNs and achieve good parallel efficiency and memory scalability while using large GPU clusters.…”

Section: Introductionmentioning

confidence: 99%

SUGAR: Efficient Subgraph-level Training via Resource-aware Graph Partitioning

Xue¹,

Yang²,

Yang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Graph Neural Networks (GNNs) have demonstrated a great potential in a variety of graph-based applications, such as recommender systems, drug discovery, and object recognition. Nevertheless, resource-efficient GNN learning is a rarely explored topic despite its many benefits for edge computing and Internet of Things (IoT) applications. To improve this state of affairs, this work proposes efficient subgraph-level training via resourceaware graph partitioning (SUGAR). SUGAR first partitions the initial graph into a set of disjoint subgraphs and then performs local training at the subgraph-level. We provide a theoretical analysis and conduct extensive experiments on five graph benchmarks to verify its efficacy in practice. Our results show that SUGAR can achieve up to 33× runtime speedup and 3.8× memory reduction on large-scale graphs. We believe SUGAR opens a new research direction towards developing GNN methods that are resource-efficient, hence suitable for IoT deployment.

show abstract

DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphs

Cited by 138 publications

References 7 publications

Survey on Graph Neural Network Acceleration: An Algorithmic Perspective

Survey on Graph Neural Network Acceleration: An Algorithmic Perspective

OpenGraphGym: A Parallel Reinforcement Learning Framework for Graph Optimization Problems

SUGAR: Efficient Subgraph-level Training via Resource-aware Graph Partitioning

Contact Info

Product

Resources

About