Neural Network-Hardware Co-design for Scalable RRAM-based BNN Accelerators

Abstract

Recently, RRAM-based Binary Neural Network (BNN) hardware has been gaining interests as it requires 1-bit sense-amp only and eliminates the need for high-resolution ADC and DAC. However, RRAM-based BNN hardware still requires high-resolution ADC for partial sum calculation to implement large-scale neural network using multiple memory arrays. We propose a neural network-hardware co-design approach to split input to fit each split network on a RRAM array so that the reconstructed BNNs calculate 1-bit output neuron in each array. As a result, ADC can be completely eliminated from the design even for large-scale neural network. Simulation results show that the proposed network reconstruction and retraining recovers the inference accuracy of the original BNN. The accuracy loss of the proposed scheme in the CIFAR-10 testcase was less than 1.1% compared to the original network. The code for training and running proposed BNN models is available at: https://github.com/YulhwaKim/RRAMScalable BNN.

STT-BNN: A Novel STT-MRAM In-Memory Computing Macro for Binary Neural Networks

“…Compared to the STT-BNN design at room temperature, the classification accuracy of the MLP slightly drops by up to 0.1% at 85 oC, whereas it drops more significantly by 3.41% for the CNN in view of its more complex task. Also, compared to the baseline BNN in [31], the classification accuracy of the MLP drops by up to 0.48% at 85 oC, it drops by 12% for the CNN. Overall, the effect of mismatch is markedly more pronounced than temperature.…”

“…Fig. 12 plots the energy per row-wise accumulation, and the inference accuracies as evaluated from two neural networks built in Torch7 [31], and mapped into the proposed STT-BNN macro similar to [8]. The first and simpler neural network is a multi-layer perceptron (MLP) for handwritten digit classification on the MNIST dataset [32], whose structure is detailed in Table II.…”

PR-CIM: a Variation-Aware Binary-Neural-Network Framework for Process-Resilient Computation-in-memory

“…should be accumulated to complete the matrix computation. SOTA works showed that the precision of the partial sums significantly affects the accuracy of the computed BNNs [2], [10], [11], [16]. To obtain multi-bit partial sums in the SRAM CIM circuits, we need ADCs to produce multi-bit outputs, incurring large area and energy overhead.…”

“…To address this problem, the authors of [11] developed an input splitting technique, which we employed in this work. A large convolution or FC layer is reconstructed into several smaller groups, as shown in Fig.…”

“…The first layer of the BNN that processes the input image and the last layer of the BNN that computes the score of each class are excluded from the input splitting, computed by digital machines. Such an approach follows SOTA works [6]- [8], [11] related to BNN. The split BNN accuracies of Table I are considered as the baselines when our techniques are evaluated.…”