GAAF: Searching Activation Functions for Binary Neural Networks through Genetic Algorithm

Li, Yanfei; Geng, Tao; Stein, S. R.; Li, Ang; Yu, Huimin

doi:10.48550/arxiv.2206.03291

Cited by 1 publication

(1 citation statement)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Besides, Q. H. Vo et al [107] presented Deepbit searching algorithm to assess the optimum BNN architecture based on the hardware cost estimation regarding the implementation target platforms. While in [104], the authors used the genetic algorithm for searching for the ideal activation functions for BNN.…”

Section: C: Gradient Error Minimizationmentioning

confidence: 99%

A Systematic Literature Review on Binary Neural Networks

et al. 2023

View full text Add to dashboard Cite

This paper presents an extensive literature review on Binary Neural Network (BNN). BNN utilizes binary weights and activation function parameters to substitute the full-precision values. In digital implementations, BNN replaces the complex calculations of Convolutional Neural Networks (CNNs) with simple bitwise operations. BNN optimizes large computation and memory storage requirements, which leads to less area and power consumption compared to full-precision models. Although there are many advantages of BNN, the binarization process has a significant impact on the performance and accuracy of the generated models. To reflect the state-of-the-art in BNN and explore how to develop and improve BNNbased models, we conduct a systematic literature review on BNN with data extracted from 239 research studies. Our review discusses various BNN architectures and the optimization approaches developed to improve their performance. There are three main research directions in BNN: accuracy optimization, compression optimization, and acceleration optimization. The accuracy optimization approaches include quantization error reduction, special regularization, gradient error minimization, and network structure. The compression optimization approaches combine fractional BNN and pruning. The acceleration optimization approaches comprise computing in-memory, FPGA-based implementations, and ASIC-based implementations. At the end of our review, we present a comprehensive analysis of BNN applications and their evaluation metrics. Also, we shed some light on the most common BNN challenges and the future research trends of BNN.

show abstract

Section: C: Gradient Error Minimizationmentioning

confidence: 99%