A Product Engine for Energy-Efficient Execution of Binary Neural Networks Using Resistive Memories

Abstract

The need for running complex Machine Learning (ML) algorithms, such as Convolutional Neural Networks (CNNs), in edge devices, which are highly constrained in terms of computing power and energy, makes it important to execute such applications efficiently. The situation has led to the popularization of Binary Neural Networks (BNNs), which significantly reduce execution time and memory requirements by representing the weights (and possibly the data being operated) using only one bit. Because approximately 90% of the operations executed by CNNs and BNNs are convolutions, a significant part of the memory transfers consists of fetching the convolutional kernels. Such kernels are usually small (e.g., 3×3 operands), and particularly in BNNs redundancy is expected. Therefore, equal kernels can be mapped to the same memory addresses, requiring significantly less memory to store them. In this context, this paper presents a custom Binary Dot Product Engine (BDPE) for BNNs that exploits the features of Resistive Random-Access Memories (RRAMs). This new engine allows accelerating the execution of the inference phase of BNNs. The novel BDPE locally stores the most used binary weights and performs binary convolution using computing capabilities enabled by the RRAMs. The system-level gem5 architectural simulator was used together with a C-based ML framework to evaluate the system's performance and obtain power results. Results show that this novel BDPE improves performance by 11.3%, energy efficiency by 7.4% and reduces the number of memory accesses by 10.7% at a cost of less than 0.3% additional die area, when integrated with a 28 nm Fully Depleted Silicon On Insulator ARMv8 in-order core, in comparison to a fully-optimized baseline of YoloV3 XNOR-Net running in a unmodified Central Processing Unit.

ALPINE: Analog In-Memory Acceleration with Tight Processor Integration for Deep Learning

“…Most closely related to our approach is the paper of Vieira et al, which details a full-system evaluation strategy of AIMC acceleration. As in our case, the authors also base their approach on AIMC-dedicated extensions to the gem5 environment [23]. Nonetheless, their approach is limited to modelling the simple case of binary CNNs, and their perkernel mapping strategy does not scale to the larger and more general applications we tackle in this paper.…”

“…6. By itself, this allows for the system-level implementation of classic loosely-coupled AIMC-enabled systems.To simulate the tightly-coupled AIMC-enabled architectures, we extend the accelerator modeling in [23] such that the custom ARMv8 ISA extension can also interface peripheral I/O (PIO) devices like our wrapper object. For this, we add connections between the ISA extension and PIO device via the system object (e.g., the simulated system that is instantiated on gem5-X's launch).…”

kNN-STUFF: kNN STreaming Unit for Fpgas

“…Xilinx Vivado reports that the floatingpoint arithmetic units responsible for computing the distances account for 12% of the in-chip energy consumption. Furthermore, the memory accesses dominate the energy consumption of the entire system, which can be as high as 90% of the total energy consumption [24]. Consequently, the units responsible for computing the distance account for only 1% of the total energy consumption.…”

Deep learning acceleration: from quantization to in-memory computing

“…Moreover, adding logic gates to the SA can realize more complex functions like addition. ReRAM IMCE [117] SOT-MRAM CA-PIM [8] SOT-MRAM BNN XNOR + popcnt PSA-BNN [118] SRAM SRAM-CIM [65] XNORAM [119] XNOR-SRAM [69] CIM-SR [120] SRAM and ReRAM XNOR-BNN [113] ReRAM-BNN [64] ReRAM FPSA-BNN [121] BDPE [122] 2T2R-TCAM [66] VR-XNOR [123] Memristor EEIM-BNN [124] SOT-MRAM MLC-CIM [125] STT-MRAM PIMBALL [126] TNN Ternary Multiplication or Gated-XNOR + popcnt TiM-DNN [68] SRAM-TPC XNOR-SRAM [69] SRAM TeC-Cell [127] FeRAM 4T2R-IM-DP [128] ReRAM SpinLiM [129] SOT-MRAM Ter-LiM [130] Memristor IMC-CD-TNN [70] Switch-Capacitor BWN Dense Addition ParaPIM [4] SOT-MRAM MRIMA [5] STT-MRAM TWN Sparse Addition Proposed FAT [9] STT-MRAM AdderNet Dense Add+Sub Proposed iMAD [131] STT-MRAM IMC accelerators for BNNs receive great research efforts thanks to BNNs' simple computation workflow. BNNs replace the 1-bit multiplication with XNOR and the 1-bit accumulation with popcnt (count the number of "1"s in a binary value) in the binary dot product.…”

“…FPSA-BNN [121] uses (+1, -1) for weights and (+1, 0) for input neurons so that they can fuse the XNOR, popcnt, and sign function to create a Fully Parallel RRAM Synaptic Array (FPSA), achieving high parallelism by reading out several consecutive rows simultaneously. BDPE [122] integrates Binary Dot Product Engine (BDPE) inside CPU for fast and energy-efficient XNOR and popcnt operations utilizing ReRAM. 2T2R-TCAM [66] creates a 2-transistor-2-ReRAM (2T2R) Ternary Content Addressable Memory (TCAM) that supports in-memory logic and XNOR/XOR-based binary dot product.…”