Binary Graph Neural Networks

Bahri, Mehdi; Bahl, Gaétan; Zafeiriou, Stefanos

doi:10.1109/cvpr46437.2021.00937

Cited by 32 publications

(19 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• Model quantification: as a particular quantification technique, binarization skillfully compresses model parameters and graph features in GNNs to yield significant acceleration in inference. Binarized DGCNN [Bahri et al, 2021] and Bi-GCN [Wang et al, 2021a] similarly introduce binarization strategies into GNNs to speed up model execution and reduce memory consumption. Degree-quant [Tailor et al, 2021] proposes a quantization-aware training method for GNNs to enable model inference with low precision integer (INT8) arithmetic, achieving up to 4.7× speedup on CPU platform.…”

Section: Gnn Compressionmentioning

confidence: 99%

“…A dynamic graph, however, is more flexible in topology and feature spaces than a static one, making it hard to apply these methods to dynamic graphs directly. Compression methods like KD-GCN and Binarized DGCNN [Bahri et al, 2021] utilize a special designed module to extend the use to dynamic graphs, providing an exemplar of dynamic graphs acceleration.…”

Section: Summary and Future Prospectsmentioning

confidence: 99%

See 1 more Smart Citation

Survey on Graph Neural Network Acceleration: An Algorithmic Perspective

Liu¹,

Yan²,

Deng³

et al. 2022

Preprint

View full text Add to dashboard Cite

Graph neural networks (GNNs) have been a hot spot of recent research and are widely utilized in diverse applications. However, with the use of huger data and deeper models, an urgent demand is unsurprisingly made to accelerate GNNs for more efficient execution. In this paper, we provide a comprehensive survey on acceleration methods for GNNs from an algorithmic perspective. We first present a new taxonomy to classify existing acceleration methods into five categories. Based on the classification, we systematically discuss these methods and highlight their correlations. Next, we provide comparisons from aspects of the efficiency and characteristics of these methods. Finally, we suggest some promising prospects for future research.

show abstract

Section: Gnn Compressionmentioning

confidence: 99%

Section: Summary and Future Prospectsmentioning

confidence: 99%

Survey on Graph Neural Network Acceleration: An Algorithmic Perspective

Liu¹,

Yan²,

Deng³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…proposed a GNN-tailored quantization algorithm, and used an automatic bit-selecting approach to pinpoint the most appropriate quantization bits. and Bahri et al (2020) further proposed binarized GNNs. Tailor et al (2021) proposed Degree-Quant, an architectureagnostic method for quantization-aware training on graphs.…”

Section: Quantizationmentioning

confidence: 99%

A Survey on Green Deep Learning

Xu¹,

Zhou²,

Fu³

et al. 2021

Preprint

View full text Add to dashboard Cite

In recent years, larger and deeper models are springing up and continuously pushing state-of-the-art (SOTA) results across various fields like natural language processing (NLP) and computer vision (CV). However, despite promising results, it needs to be noted that the computations required by SOTA models have been increased at an exponential rate. Massive computations not only have a surprisingly large carbon footprint but also have negative effects on research inclusiveness and deployment on real-world applications. Green deep learning is an increasingly hot research field that appeals to researchers to pay attention to energy usage and carbon emission during model training and inference. The target is to yield novel results with lightweight and efficient technologies. Many technologies can be used to achieve this goal, like model compression and knowledge distillation. This paper focuses on presenting a systematic review of the development of Green deep learning technologies. We classify these approaches into four categories: (1) compact networks, (2) energy-efficient training strategies, (3) energy-efficient inference approaches, and (4) efficient data usage. For each category, we discuss the progress that has been achieved and the unresolved challenges.

show abstract

“…[49] applied bucketbased quantization of matrix-matrix products to accelerate the GCN [20] operator. [4] proposed a general framework for binarizing graph neural networks and specifically introduced efficient binarized versions of the Dynamic Edge-Conv operator [50] with real-world speed-ups on a lowpower device.…”

Section: Relational Inference and Gnnsmentioning

confidence: 99%

“…We benchmark our method against state-of-the-art approaches on the RoadTracer [5] dataset and find that it is orders of magnitude faster than recent competing methods while retaining competitive accuracy. With the increasing interest in deep learning inference on embedded hardware, such as satellites or drones [3], our method, combined with recent advances in CNN and GNN quantization [4,8], opens the possibility of on board in-flight road extraction, which could reduce ground computation and bandwidth needs by transmitting graphs instead of images.…”

Section: Introductionmentioning

confidence: 99%

Road Extraction from Overhead Images with Graph Neural Networks

Bahl¹,

Bahri²,

Lafarge³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Automatic road graph extraction from aerial and satellite images is a long-standing challenge. Existing algorithms are either based on pixel-level segmentation followed by vectorization, or on iterative graph construction using next move prediction. Both of these strategies suffer from severe drawbacks, in particular high computing resources and incomplete outputs. By contrast, we propose a method that directly infers the final road graph in a single pass. The key idea consists in combining a Fully Convolutional Network in charge of locating points of interest such as intersections, dead ends and turns, and a Graph Neural Network which predicts links between these points. Such a strategy is more efficient than iterative methods and allows us to streamline the training process by removing the need for generation of starting locations while keeping the training end-to-end. We evaluate our method against existing works on the popular RoadTracer dataset and achieve competitive results. We also benchmark the speed of our method and show that it outperforms existing approaches. This opens the possibility of in-flight processing on embedded devices.

show abstract

Binary Graph Neural Networks

Cited by 32 publications

References 25 publications

Survey on Graph Neural Network Acceleration: An Algorithmic Perspective

Survey on Graph Neural Network Acceleration: An Algorithmic Perspective

A Survey on Green Deep Learning

Road Extraction from Overhead Images with Graph Neural Networks

Contact Info

Product

Resources

About