A Study of Array Resistance Distribution and a Novel Operation Algorithm for WO&lt;sub&gt;x&lt;/sub&gt; ReRAM Memory

Hsu, K.C.; Lee, F.M.; Lin, Yu-Yu; Lai, Erh-Kun; Wu, Jin-Shang; Lee, D.Y.; Lee, M.H.; Lung, Hsiang-Lan; Hsieh, Kuang-Yeu; Lu, Ching-Sen

doi:10.7567/ssdm.2015.o-3-5

Cited by 14 publications

(10 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, compared with GPUs that perform computation in high precision, the computation in an RRAM device is limited by finite quantization levels. Furthermore, RRAM IEEE Journal on Exploratory Solid-State Computational Devices and Circuits operations in the analog domain [12] is associated with several challenges, such as high device-to-device variation [13], [14], stuck-at-faults (SAFs) [15], and limited ON/OFF ratio. The stochastic process variations in RRAM cause deviation in the programmed conductance value from the pretrained value, resulting in incorrect weight in the model.…”

Section: Introductionmentioning

confidence: 99%

Accurate Inference With Inaccurate RRAM Devices: A Joint Algorithm-Design Solution

Charan

Mohanty

et al. 2020

IEEE J. Explor. Solid-State Comput. Devices Circuits

View full text Add to dashboard Cite

Resistive random access memory (RRAM) is a promising technology for energy-efficient neuromorphic accelerators. However, when a pretrained deep neural network (DNN) model is programmed to an RRAM array for inference, the model suffers from accuracy degradation due to RRAM nonidealities, such as device variations, quantization error, and stuck-at-faults. Previous solutions involving multiple readverify-write (R-V-W) to the RRAM cells require cell-by-cell compensation and, thus, an excessive amount of processing time. In this article, we propose a joint algorithm-design solution to mitigate the accuracy degradation. We first leverage knowledge distillation (KD), where the model is trained with the RRAM nonidealities to increase the robustness of the model under device variations. Furthermore, we propose random sparse adaptation (RSA), which integrates a small on-chip memory with the main RRAM array for postmapping adaptation. Only the on-chip memory is updated to recover the inference accuracy. The joint algorithm-design solution achieves the state-of-the-art accuracy of 99.41% for MNIST (LeNet-5) and 91.86% for CIFAR-10 (VGG-16) with up to 5% parameters as overhead while providing a 15-150× speedup compared with R-V-W. INDEX TERMS Convolution neural networks, device nonidealities, model robustness, neuromorphic computing, random sparse adaptation (RSA), resistive random access memory (RRAM).

show abstract

Section: Introductionmentioning

confidence: 99%

Accurate Inference With Inaccurate RRAM Devices: A Joint Algorithm-Design Solution

Charan

Mohanty

et al. 2020

IEEE J. Explor. Solid-State Comput. Devices Circuits

View full text Add to dashboard Cite

show abstract

“…This resistance variation of ReRAM greatly afects the error rate of matrix-vector multiplications. Recent work shows that the resistance histograms of both LRS and HRS follow the lognormal distribution [11]. As shown in Figure 4(b), the resistance distribution is mainly determined by the mean resistance of the on-state (R on ), the resistance ratio between on and of state (R-ratio = R of f /R on ), and the resistance deviation (σ ) of each state.…”

Section: Reram Cellmentioning

confidence: 91%

“…Figure 12 shows the simulation low of this module. First, given the number of bits per cell, R on , R-ratio, and resistance deviation (σ ), we can build the cell resistance distribution, which is modeled by the Probability Density Function (PDF) of lognormal distribution [11], from low-resistance to high-resistance states:…”

Section: Nvm Error Analytical Modulementioning

confidence: 99%

“…We then perform shift-and-add based on the bit-position of the feature map, followed by the shift-and-add based on the bit-position of the ilter. The example in Figure 6(e) shows that, by applying the Composition process, we can resemble the OU-based bit-level outputs (1,1), (0, 0), (0,1), (1,1), into the inal error-included result (11). The error-injected and recomposed output feature maps are then transformed from ixed-point to loating-point representation and passed to the next TensorFlow pooling layer.…”

Section: Tensorflow-based Inference Accuracy Simulation Modulementioning

confidence: 99%

See 1 more Smart Citation

DL-RSIM: A Reliability and Deployment Strategy Simulation Framework for ReRAM-based CNN Accelerators

Lin

Cheng

Yang

et al. 2022

ACM Trans. Embed. Comput. Syst.

View full text Add to dashboard Cite

Memristor-based deep learning accelerators provide a promising solution to improve the energy efficiency of neuromorphic computing systems. However, the electrical properties and crossbar structure of memristors make these accelerators error-prone. In addition, due to the hardware constraints, the way to deploy neural network models on memristor crossbar arrays affects the computation parallelism and communication overheads.To enable reliable and energy-efficient memristor-based accelerators, a simulation platform is needed to precisely analyze the impact of non-ideal circuit/device properties on the inference accuracy and the influence of different deployment strategies on performance and energy consumption. In this paper, we propose a flexible simulation framework, DL-RSIM, to tackle this challenge. A rich set of reliability impact factors and deployment strategies are explored by DL-RSIM, and it can be incorporated with any deep learning neural networks implemented by TensorFlow. Using several representative convolutional neural networks as case studies, we show that DL-RSIM can guide chip designers to choose a reliability-friendly design option and energy-efficient deployment strategies and develop optimization techniques accordingly.

show abstract

“…This formation of conductive lament involves randomness, and therefore even with extensive program/read scheme, some variation in the resistance of RRAM cell is inevitable [11]. It has been reported that, for both HRS and LRS cells, the resistance follows the lognormal distribution characterized by the resistance value and resistance deviation for each state [12]. Such statistical randomness in cell resistance leads to a deviation of bitline current.…”

Section: Challenges In Exploiting Wordline-level Parallelismmentioning

confidence: 99%

Unlocking wordline-level parallelism for fast inference on RRAM-based DNN accelerator

Park

Lee

Shin

et al. 2020

Proceedings of the 39th International Conference on Computer-Aided Design

View full text Add to dashboard Cite

A Study of Array Resistance Distribution and a Novel Operation Algorithm for WO<sub>x</sub> ReRAM Memory

Cited by 14 publications

References 0 publications

Accurate Inference With Inaccurate RRAM Devices: A Joint Algorithm-Design Solution

Accurate Inference With Inaccurate RRAM Devices: A Joint Algorithm-Design Solution

DL-RSIM: A Reliability and Deployment Strategy Simulation Framework for ReRAM-based CNN Accelerators

Unlocking wordline-level parallelism for fast inference on RRAM-based DNN accelerator

Contact Info

Product

Resources

About