Deep learning binary neural network on an FPGA

Zhou, Yuteng; Redkar, Shrutika; Huang, Xinming

doi:10.1109/mwscas.2017.8052915

Cited by 22 publications

(16 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[53] 86.0 ResNet-18 [64] 86.05 9 256-ch conv. [61] 86.06 5 conv. and 2 FC [65] 86.78 NK [35] 86.98 C64-MP-2C128-MP-2C256-2FC512-FC10 [51] 87 NK [34] 87.16 AlexNet R1 regularizer [34] 87.30 AlexNet R2 regularizer [38] 87.73 BNN +1 padding [32] 88 BNN 512 channels for FC [38] 88.42 BNN 0 padding [59] 88.47 6 conv.…”

Section: Source Accuracy (%) Topology Precisionmentioning

confidence: 99%

See 1 more Smart Citation

A Review of Binarized Neural Networks

2019

View full text Add to dashboard Cite

In this work, we review Binarized Neural Networks (BNNs). BNNs are deep neural networks that use binary values for activations and weights, instead of full precision values. With binary values, BNNs can execute computations using bitwise operations, which reduces execution time. Model sizes of BNNs are much smaller than their full precision counterparts. While the accuracy of a BNN model is generally less than full precision models, BNNs have been closing accuracy gap and are becoming more accurate on larger datasets like ImageNet. BNNs are also good candidates for deep learning implementations on FPGAs and ASICs due to their bitwise efficiency. We give a tutorial of the general BNN methodology and review various contributions, implementations and applications of BNNs.

show abstract

Section: Source Accuracy (%) Topology Precisionmentioning

confidence: 99%

“…CPU-FPGA hybrid systems offer a CPU and FPGA connected in the same silicon. These systems are widely used to implement DNNs and BNNs [35,38,39,41,54,[61][62][63]68,71]. The CPU is flexible and easily programmed to load inputs to the DNN.…”

Section: Architecturesmentioning

confidence: 99%

A Review of Binarized Neural Networks

2019

View full text Add to dashboard Cite

show abstract

“…A neural network which uses any combination of binary weight or hard threshold activation functions is typically known as BNN. There have been several successful implementations of BNN algorithms in software [26], [27], [28], [29] and an attempt to implement it on hardware [30], [31], [32]. The analog hardware implementation of BNN system with learning remains as an open problem [33], [32].…”

Section: Background a Learning Algorithms And Biologically Inspimentioning

confidence: 99%

“…Section V contains the circuit and system level simulation results. Section VI discusses advantages and limitations of the proposed circuits and introduces the aspects of the arXiv:1808.10631v1 [cs.ET] 31 Aug 2018 design that should be investigated in future, and Section VII concludes the paper. There is also a supplementary Material that includes the expanded background information, explicit explanation of the proposed circuit, the device parameters of the main backpropagation circuit proposed in Section III and simulation results for the learning circuit performance.…”

Section: Introductionmentioning

confidence: 99%

Learning in Memristive Neural Network Architectures Using Analog Backpropagation Circuits

Krestinskaya

Salama

James

2019

IEEE Trans. Circuits Syst. I

128

View full text Add to dashboard Cite

The on-chip implementation of learning algorithms would speed-up the training of neural networks in crossbar arrays. The circuit level design and implementation of backpropagation algorithm using gradient descent operation for neural network architectures is an open problem. In this paper, we proposed the analog backpropagation learning circuits for various memristive learning architectures, such as Deep Neural Network (DNN), Binary Neural Network (BNN), Multiple Neural Network (MNN), Hierarchical Temporal Memory (HTM) and Long-Short Term Memory (LSTM). The circuit design and verification is done using TSMC 180nm CMOS process models, and TiO2 based memristor models. The application level validations of the system are done using XOR problem, MNIST character and Yale face image databases.

show abstract

“…Notably, early approaches for the BNN models in Courbariaux et al (2015Courbariaux et al ( , 2016; Rastegari et al (2016) quantize weights or activations into {+1, −1}, which replaces floating-point multiplications with binary bitwise operations, thus approximating the floating-point multiply-accumulate operation using bitwise XNOR and bit counting operations. Besides, the quantized binary weights can reduce weight storage requirements, which makes BNNs a highly appealing method for implementing CNNs on embedded systems and programmable devices (Guo, 2018;Zhou et al, 2017b;Yi et al, 2018;Liang et al, 2018). Despite many benefits above, the low precision of the binarized operations in BNNs degrades the classification ability on modern CNNs, thus limiting their applications.…”

Section: Introductionmentioning

confidence: 99%

Peer Review #1 of "AresB-Net: accurate residual binarized neural networks using shortcut concatenation and shuffled grouped convolution (v0.1)"

2021

View full text Add to dashboard Cite

This paper proposes a novel network model to achieve better accurate residual binarized convolutional neural networks (CNNs), denoted as AresB-Net. Even though residual CNNs enhance the classification accuracy of binarized neural networks (BNNs) with increasing feature resolution, the degraded classification accuracy is still the primary concern compared with real-valued residual CNNs. AresB-Net consists of novel basic blocks to amortize the severe error from the binarization, suggesting a well-balanced pyramid structure without downsampling convolution. In each basic block, the shortcut is added to the convolution output and then concatenated, and then the expanded channels are shuffled for the next grouped convolution. In the downsampling when stride > 1, our model adopts only the max-pooling layer for generating low-cost shortcut. This structure facilitates the feature reuse from the previous layers, thus alleviating the error from the binarized convolution and increasing the classification accuracy with reduced computational costs and small weight storage requirements. Despite low hardware costs from the binarized computations, the proposed model achieves remarkable classification accuracies on the CIFAR and ImageNet datasets.

show abstract

Deep learning binary neural network on an FPGA

Cited by 22 publications

References 4 publications

A Review of Binarized Neural Networks

A Review of Binarized Neural Networks

Learning in Memristive Neural Network Architectures Using Analog Backpropagation Circuits

Peer Review #1 of "AresB-Net: accurate residual binarized neural networks using shortcut concatenation and shuffled grouped convolution (v0.1)"

Contact Info

Product

Resources

About