XNOR-RRAM: A scalable and parallel resistive synaptic architecture for binary neural networks

Sun, Xiaoyu; Yin, Shihui; Peng, Xiaochen; Liu, Rui; Seo, Jae-sun; Yu, Shimeng

doi:10.23919/date.2018.8342235

Cited by 184 publications

(115 citation statements)

References 12 publications

Supporting

Mentioning

115

Contrasting

Order By: Relevance

“…Topology [52] 95.7 FC200-3FC100-FC10 [1] 96.0 MLP [51] 97 NK [53] 97.0 LeNet [54] 97.69 MLP [55] 97.86 ConvPool-2 [35] 98. 25 1/4 MLP [41] 98.4 MLP [56] 98.40 MLP [57] 98.6 NK [58] 98.67 MLP [59] 98.77 FC784-3FC512-FC10 Table 6. Accuracies on the MNIST dataset of non-binary networks related to works reviewed.…”

Section: Source Accuracy (%)mentioning

confidence: 99%

“…[61] 86.06 5 conv. and 2 FC [65] 86.78 NK [35] 86.98 C64-MP-2C128-MP-2C256-2FC512-FC10 [51] 87 NK [34] 87.16 AlexNet R1 regularizer [34] 87.30 AlexNet R2 regularizer [38] 87.73 BNN +1 padding [32] 88 BNN 512 channels for FC [38] 88.42 BNN 0 padding [59] 88.47 6 conv. [39] 88.61 NK [1] 89.85 BNN Table 10.…”

Section: Source Accuracy (%) Topology Precisionmentioning

confidence: 99%

“…Custom silicon also allows for the use of mixed technologies and memory based designs in resistive RAM (RRAM). RRAM is an emerging technology and is an appealing platform for BNN designs due to its compact operation at the bit level [59,60,77,81,82].…”

Section: Asicsmentioning

confidence: 99%

See 2 more Smart Citations

A Review of Binarized Neural Networks

2019

View full text Add to dashboard Cite

In this work, we review Binarized Neural Networks (BNNs). BNNs are deep neural networks that use binary values for activations and weights, instead of full precision values. With binary values, BNNs can execute computations using bitwise operations, which reduces execution time. Model sizes of BNNs are much smaller than their full precision counterparts. While the accuracy of a BNN model is generally less than full precision models, BNNs have been closing accuracy gap and are becoming more accurate on larger datasets like ImageNet. BNNs are also good candidates for deep learning implementations on FPGAs and ASICs due to their bitwise efficiency. We give a tutorial of the general BNN methodology and review various contributions, implementations and applications of BNNs.

show abstract

Section: Source Accuracy (%)mentioning

confidence: 99%

Section: Source Accuracy (%) Topology Precisionmentioning

confidence: 99%

See 1 more Smart Citation

A Review of Binarized Neural Networks

2019

View full text Add to dashboard Cite

show abstract

“…In each of these works the resistive element is used to perform simple logical operations based on current sensing technique. In [26,27,30,31] several Binary Convolutional Neural Networks (BCNNs) implementations are discussed: they achieve very good results in terms of energy and power, thanks to the intrinsic low power nature of the MTJ and RRAM devices. Reference [28] proposes a BNN design based on SRAM array.…”

Section: Nn Implementations Based On Lim Conceptmentioning

confidence: 99%

Logic-in-Memory Computation: Is It Worth It? A Binary Neural Network Case Study

Coluccio

Vacca

Turvani

2020

JLPEA

View full text Add to dashboard Cite

Recently, the Logic-in-Memory (LiM) concept has been widely studied in the literature. This paradigm represents one of the most efficient ways to solve the limitations of a Von Neumann’s architecture: by placing simple logic circuits inside or near a memory element, it is possible to obtain a local computation without the need to fetch data from the main memory. Although this concept introduces a lot of advantages from a theoretical point of view, its implementation could introduce an increasing complexity overhead of the memory itself, leading to a more sophisticated design flow. As a case study, Binary Neural Networks (BNNs) have been chosen. BNNs binarize both weights and inputs, transforming multiply-and-accumulate into a simpler bitwise logical operation while maintaining high accuracy, making them well-suited for a LiM implementation. In this paper, we present two circuits implementing a BNN model in CMOS technology. The first one, called Out-Of-Memory (OOM) architecture, is implemented following a standard Von Neumann structure. The same architecture was redesigned to adapt the critical part of the algorithm for a modified memory, which is also capable of executing logic calculations. By comparing both OOM and LiM architectures we aim to evaluate if Logic-in-Memory paradigm is worth it. The results highlight that LiM architectures have a clear advantage over Von Neumann architectures, allowing a reduction in energy consumption while increasing the overall speed of the circuit.

show abstract

“…Previous works on ReRAM based accelerators for BNNs, such as Sun et al (2018b), Tang et al (2017), failed to perform real in-situ computation in the ReRAM. ReRAM arrays are only used as processing engines for low precision multiplication in Sun et al (2018b, Tang et al (2017, and extra logic components are deployed for the nonlinear activation and binary quantization. In this work, we enables the real in-situ computation in ReRAM for BNN processing through a software/hardware codesign approach, including (1) the modified hardware oriented operation flow and (2) the fused activation with CRC array.…”

Section: Fused Activation With Crc Arraymentioning

confidence: 99%

ReBNN: in-situ acceleration of binarized neural networks in ReRAM using complementary resistive cell

Song

Qian

et al. 2019

CCF Trans. HPC

View full text Add to dashboard Cite

Resistive random access memory (ReRAM) has been proven capable to efficiently perform in-situ matrix-vector computations in convolutional neural network (CNN) processing. The computations are often conducted on multi-level cell (MLC) that have limited precision and hence, show significant vulnerability to noises. The binarized neural network (BNN) is a hardware-friendly model that can dramatically reduce the computation and storage overheads. However, XNOR, which is the key operation in BNNs, cannot be directly computed in-situ in ReRAM because of its nonlinear behavior. To enable real insitu processing of BNNs in ReRAM, we modified the BNN algorithm to enable direct computation of XNOR, POPCOUNT and POOL based on ReRAM cells. We also proposed the complementary resistive cell (CRC) design to efficiently conduct XNOR operations and optimized the pipeline design with decoupled buffer and computation stages. Our results show our scheme, namely, ReBNN, improves the system performance by 25.36× and the energy efficiency by 4.26× compared to conventional ReRAM based accelerator, and ensures a throughput higher than state-of-the-art BNN accelerators. The correctness of the modified algorithm is also validated.

show abstract

XNOR-RRAM: A scalable and parallel resistive synaptic architecture for binary neural networks

Cited by 184 publications

References 12 publications

A Review of Binarized Neural Networks

A Review of Binarized Neural Networks

Logic-in-Memory Computation: Is It Worth It? A Binary Neural Network Case Study

ReBNN: in-situ acceleration of binarized neural networks in ReRAM using complementary resistive cell

Contact Info

Product

Resources

About