A Review of Binarized Neural Networks

Simons, Taylor; Lee, Dah-Jye

doi:10.3390/electronics8060661

Cited by 193 publications

(140 citation statements)

References 58 publications

Supporting

Mentioning

135

Contrasting

Unclassified

Order By: Relevance

“…Memristor crossbar circuit is then simulated by Cadence Spectre circuit simulation [16]. For the binary neural network, the crossbar circuit is trained using by the modified version of back-propagation algorithm [17]. The obtained synaptic weight are converted to the memristance values using Equation 2.…”

Section: Resultsmentioning

confidence: 99%

Optimizing the Distribution of Memristance Values of Memristive Synapses for Reducing Power Consumption in Analog Memristor Crossbar Based Neural Networks

Truong

2019

IJEAT

View full text Add to dashboard Cite

Memristor circuits have become one of the potential hardware-based platforms for implementing artificial neural networks due to a lot of advantageous features. In this paper, we compare the power consumption between an analog memristor crossbar-based a binary memristor crossbar-based neural network for realizing a two-layer neural network and propose an efficient method for reducing the power consumption of the analog memristor crossbar-based neural network. A two-layer neural network is implemented using the memristor crossbar arrays, which can be used with analog synapse or binary synapse. For recognizing the test samples of MNIST dataset, the binary memristor crossbar-based neural work consumes higher power by 19% than the analog memristor-based neural network. The power consumption of the analog memristor crossbar-based neural network strongly depends on the distribution of memristance values and it can be reduced by optimizing the distribution of the memristance values. To improve the power efficiency, the bias resistance must be selected close to high resistance state. The power consumption of the analog memristor-based neural network is reduced by 86% when increasing the bias resistance from 20KΩ to 160KΩ. For the bias resistance of 160KΩ, analog memristor crossbar-based neural network consumes less power by 89% than the binary memristor crossbar-based neural network.

show abstract

Section: Resultsmentioning

confidence: 99%

Optimizing the Distribution of Memristance Values of Memristive Synapses for Reducing Power Consumption in Analog Memristor Crossbar Based Neural Networks

Truong

2019

IJEAT

View full text Add to dashboard Cite

show abstract

“…Therefore, in this work, we will adopt the BNN and implement it in the FPGA device to recognize our target crops. Furthermore, due to binarization, the accuracies of the existing BNNs were also reduced compared to the CNNs with full precision [22]. To enhance the recognition accuracy of target crops using the BNN in a real scene, a thresholding scheme is also presented in this work.…”

Section: B Bnn Designsmentioning

confidence: 99%

An FPGA-Based Hardware/Software Design Using Binarized Neural Networks for Agricultural Applications: A Case Study

Huang

2021

IEEE Access

View full text Add to dashboard Cite

This work presents an FPGA-based hardware/software design to help the agricultural robot intelligently decide if biological agents need to be applied to the target crops. For target crop recognition, in our global positioning, the selective search integrates with a thresholding scheme to reduce the number of region of interest (ROI) in a captured image. In our local recognition, a binarized neural network (BNN) architecture is presented to help recognize the target crop. Furthermore, an estimation method of pest and disease severity is also presented. Experiments show integrating our presented BNN architecture needs a few extra resources (less than 17% of available FPGA resources in terms of Xilinx Zynq UltraScall+ T M MPSoC ZU3EG A484), compared to an existing BNN one. However, the top-1 accuracy rate and the top-5 one can be increased by 32.25% to 32.84% and by 14.99% to 15.17%, respectively. Furthermore, when the presented BNN architecture was also implemented on the ARM Cortex-A53 CPU and the NVIDIA GeForce RTX 2080 GPU, our BNN hardware module on the FPGA can accelerate the frames per second (FPS) by a factor of 3,690.18 and a factor of 1.07, respectively. INDEX TERMS FPGA, binarized neural networks, object detection, agriculture

show abstract

“…Compared with analog-type neural networks, digital-type neural networks are simpler, can work more efficiently, while with limited decrease in the network performances. [84][85][86]…”

Section: Nonideality Of Artificial Synapsesmentioning

confidence: 99%

Artificial Neural Networks Based on Memristive Devices: From Device to System

Huang

Wang

et al. 2020

Advanced Intelligent Systems

View full text Add to dashboard Cite

Thanks to the advance in transistor scaling, computing capabilities of central processing units (CPUs) and graphics processing units (GPUs) have been increased greatly, resulting in the fast development of artificial neural networks (ANNs). ANNs predict the correlation between a set of input and output parameters by emulating the principles of biological neural networks. [1] Nowadays, ANNs have been applied to a broad spectrum, such as computer vision, natural language processing, autonomous driving, and decision making. Although ANNs are advantageous in some specific tasks, such as playing the game of Go, [2,3] the human brain outperforms ANNs in cognitive and classification tasks with fast speed and high power efficiency. The advantages of the human brain lie in the special architectures with high massive parallelism, spiking neurons, and enormous number of basic computing units of neurons (%10 11) and synapses (%10 15), whereas computers show high speed of basic operation and precision. [4] The massively parallel processing compensates the low speed of neurons and synapses, whereas spiking neurons compute more powerful than other types of neurons. [5] The parallel processing occurs based on variable synaptic weights, which is the basis of memory. Moreover, the same information is processed by many neurons and transmitted to a downstream neuron, which enhances the precision as well. Therefore, the human brain shows a salient feature, i.e., in-memory computing, suitable for high-performance applications with a large amount of data. In contrast, conventional digital computing systems face the Von Neumann bottleneck, because in the Von Neumann architecture, computing units and memory are separated, hindering the enhancement of ANNs. Thus, several novel architectures are promoted, such as brain-inspired computing (processing information based on spiking neurons, also called as spiking neural networks [SNNs]) and in-memory computing on chip, to overcome the Von Neumann bottleneck. [6] To implement the new architectures, artificial synapses and neurons are necessary. The massive parallelism of the brain takes advantage of a large number of neurons and synapses; however, the emulation of the functions of biological synapses and neurons requires tens of complement metal-oxide-semiconductor (CMOS) transistors. Therefore, it is urgent to realize synapses and neurons with a simplified method for the development of ANNs. [7] As synapses adjust their weights step-by-step repeatedly, synaptic devices need to exhibit analog switching behavior. [8] However, biological neurons show the analog membrane potential and generate the all-or-nothing rapidly, so both analog and digital devices are acceptable for the emulation of neurons. Many emerging devices have been proposed to develop ANNs based on hardware, e.g., memristive devices, phase change memories (PCMs), [9] magnetic random-access memories (MRAMs), [10] and ferroelectric field-effect transistors (FeFETs). [11] Each technology has its advantages and disadvantages. PCM and M...

show abstract

A Review of Binarized Neural Networks

Cited by 193 publications

References 58 publications

Optimizing the Distribution of Memristance Values of Memristive Synapses for Reducing Power Consumption in Analog Memristor Crossbar Based Neural Networks

Optimizing the Distribution of Memristance Values of Memristive Synapses for Reducing Power Consumption in Analog Memristor Crossbar Based Neural Networks

An FPGA-Based Hardware/Software Design Using Binarized Neural Networks for Agricultural Applications: A Case Study

Artificial Neural Networks Based on Memristive Devices: From Device to System

Contact Info

Product

Resources

About