AdaBin: Improving Binary Neural Networks with Adaptive Binary Sets

Tu, Zhijun; Chen, Xinghao; Ren, Pengju; Wang, Yunhe

doi:10.1007/978-3-031-20083-0_23

Cited by 33 publications

(12 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Competitive quantisation works endeavour to approximate model data to fewer than 4 bits. Binary neural networks perform extreme quantisation to single-bit values but suffer considerable accuracy loss in both convolution [21,27] and Transformer models [25]. Conversely, low-bit QNNs maintain high accuracy by two main techniques.…”

Section: Neural Network Quantisationmentioning

confidence: 99%

Table-Lookup MAC: Scalable Processing of Quantised Neural Networks in FPGA Soft Logic

Gerlinghoff,

Choong,

Goh

et al. 2024

Proceedings of the 2024 ACM/SIGDA International Symposium on Field Programmable Gate Arrays

View full text Add to dashboard Cite

Recent advancements in neural network quantisation have yielded remarkable outcomes, with three-bit networks reaching state-ofthe-art full-precision accuracy in complex tasks. These achievements present valuable opportunities for accelerating neural networks by computing in reduced precision. Implementing it on FPGAs can take advantage of bit-level reconfigurability, which is not available on conventional CPUs and GPUs. Simultaneously, the high data intensity of neural network processing has inspired computing-in-memory paradigms, including on FPGA platforms. By programming the effects of trained model weights as lookup operations in soft logic, the transfer of weight data from memory units can be avoided, alleviating the memory bottleneck. However, previous methods face poor scalability -the high logic utilisation limiting them to small networks/sub-networks of binary models with low accuracy. In this paper, we introduce Table Lookup Multiply-Accumulate (TLMAC) as a framework to compile and optimise quantised neural networks for scalable lookup-based processing. TLMAC clusters and maps unique groups of weights to lookupbased processing elements, enabling highly parallel computation while taking advantage of parameter redundancy. Further place and route algorithms are proposed to reduce LUT utilisation and routing congestion. We demonstrate that TLMAC significantly improves the scalability of previous related works. Our efficient logic mapping and high degree of reuse enables entire ImageNet-scale quantised models with full-precision accuracy to be implemented using lookup-based computing on one commercially available FPGA.

show abstract

Section: Neural Network Quantisationmentioning

confidence: 99%

Table-Lookup MAC: Scalable Processing of Quantised Neural Networks in FPGA Soft Logic

Gerlinghoff,

Choong,

Goh

et al. 2024

Proceedings of the 2024 ACM/SIGDA International Symposium on Field Programmable Gate Arrays

View full text Add to dashboard Cite

show abstract

“…While RB-Net [92] presented a reshaped point-wise convolution (RPC) and balanced distribution activation (BA) for a more powerful representative ability. Also, AdaBin [93] provides adaptive binary sets for weights and activations for each layer that centers the position and distance of the distribution of the binary values to real-valued distribution. Besides, INSTA-BNN [94] determines the activation threshold value based on the difference between statistical data generated from a batch and each instance to increase the accuracy.…”

Section: C: Gradient Error Minimizationmentioning

confidence: 99%

“…In addition, modifying the network structure to improve the accuracy like ReActNet-C [87], INSTA-BNN [94], [93], BCDNet-A [95], and [101]. They have an accuracy very close to the full-precision MobileNet-v2 by a difference of less than 1%.…”

Section: ) Image Classificationmentioning

confidence: 99%

A Systematic Literature Review on Binary Neural Networks

et al. 2023

View full text Add to dashboard Cite

This paper presents an extensive literature review on Binary Neural Network (BNN). BNN utilizes binary weights and activation function parameters to substitute the full-precision values. In digital implementations, BNN replaces the complex calculations of Convolutional Neural Networks (CNNs) with simple bitwise operations. BNN optimizes large computation and memory storage requirements, which leads to less area and power consumption compared to full-precision models. Although there are many advantages of BNN, the binarization process has a significant impact on the performance and accuracy of the generated models. To reflect the state-of-the-art in BNN and explore how to develop and improve BNNbased models, we conduct a systematic literature review on BNN with data extracted from 239 research studies. Our review discusses various BNN architectures and the optimization approaches developed to improve their performance. There are three main research directions in BNN: accuracy optimization, compression optimization, and acceleration optimization. The accuracy optimization approaches include quantization error reduction, special regularization, gradient error minimization, and network structure. The compression optimization approaches combine fractional BNN and pruning. The acceleration optimization approaches comprise computing in-memory, FPGA-based implementations, and ASIC-based implementations. At the end of our review, we present a comprehensive analysis of BNN applications and their evaluation metrics. Also, we shed some light on the most common BNN challenges and the future research trends of BNN.

show abstract

“…In reference [ 46 ], the AdaBin method was proposed to adaptively obtain the optimal binary set {b1, b2} for each layer of weights and activations. The method defined a new binary quantization function using the center position and distance of 1-bit values.…”

Section: The Key Technologies Of Embedded Aimentioning

confidence: 99%

A Review of Artificial Intelligence in Embedded Systems

Zhang

2023

Micromachines

View full text Add to dashboard Cite

Advancements in artificial intelligence algorithms and models, along with embedded device support, have resulted in the issue of high energy consumption and poor compatibility when deploying artificial intelligence models and networks on embedded devices becoming solvable. In response to these problems, this paper introduces three aspects of methods and applications for deploying artificial intelligence technologies on embedded devices, including artificial intelligence algorithms and models on resource-constrained hardware, acceleration methods for embedded devices, neural network compression, and current application models of embedded AI. This paper compares relevant literature, highlights the strengths and weaknesses, and concludes with future directions for embedded AI and a summary of the article.

show abstract

AdaBin: Improving Binary Neural Networks with Adaptive Binary Sets

Cited by 33 publications

References 25 publications

Table-Lookup MAC: Scalable Processing of Quantised Neural Networks in FPGA Soft Logic

Table-Lookup MAC: Scalable Processing of Quantised Neural Networks in FPGA Soft Logic

A Systematic Literature Review on Binary Neural Networks

A Review of Artificial Intelligence in Embedded Systems

Contact Info

Product

Resources

About