Performance Impacts of Analog ReRAM Non-ideality on Neuromorphic Computing

Lin, Yu–Hsuan; Wang, Chao-Hung; Lee, Ming-Hsiu; Lee, Dai-Ying; Lin, Yu-Yu; Lee, Feng-Min; Lung, Hsiang-Lan; Wang, Keh-Chung; Tseng, Tseung‐Yuen; Lu, Chih‐Yuan

doi:10.1109/ted.2019.2894273

Cited by 61 publications

(25 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The results show that the resistance can experience abrupt variations, called random walk, in addition to the typical random telegraph noise (RTN). For instance, the accuracy of neural network can decrease with time due to the introduction of a bias in the conductance as a result of the drift [43,58,60]. Figure 7c shows the cumulative distribution of the initial and drifted conductance by annealing at T = 125°C of Figure 6c [43], which confirms the time-dependent drift of weights of a two-layer neural network.…”

Section: Conductance Drift and Fluctuationsmentioning

confidence: 64%

“…Unfortunately, once the device is programmed with a given precision, the conductance might still change due to time-dependent drift and fluctuation that affects the reliability of IMC. Figure 7a shows measured traces of conductance programmed with 4 levels on a 1Mb RRAM array as a function of time during annealing at 150°C [58]. The change of conductance can be explained by the thermally-activated atomic diffusion in the conductive filament.…”

Section: Conductance Drift and Fluctuationsmentioning

confidence: 99%

See 1 more Smart Citation

In-Memory Computing with Resistive Memory Circuits: Status and Outlook

Pedretti

Ielmini

2021

Electronics

View full text Add to dashboard Cite

In-memory computing (IMC) refers to non-von Neumann architectures where data are processed in situ within the memory by taking advantage of physical laws. Among the memory devices that have been considered for IMC, the resistive switching memory (RRAM), also known as memristor, is one of the most promising technologies due to its relatively easy integration and scaling. RRAM devices have been explored for both memory and IMC applications, such as neural network accelerators and neuromorphic processors. This work presents the status and outlook on the RRAM for analog computing, where the precision of the encoded coefficients, such as the synaptic weights of a neural network, is one of the key requirements. We show the experimental study of the cycle-to-cycle variation of set and reset processes for HfO2-based RRAM, which indicate that gate-controlled pulses present the least variation in conductance. Assuming a constant variation of conductance σG, we then evaluate and compare various mapping schemes, including multilevel, binary, unary, redundant and slicing techniques. We present analytical formulas for the standard deviation of the conductance and the maximum number of bits that still satisfies a given maximum error. Finally, we discuss RRAM performance for various analog computing tasks compared to other computational memory devices. RRAM appears as one of the most promising devices in terms of scaling, accuracy and low-current operation.

show abstract

Section: Conductance Drift and Fluctuationsmentioning

confidence: 64%

Section: Conductance Drift and Fluctuationsmentioning

confidence: 99%

In-Memory Computing with Resistive Memory Circuits: Status and Outlook

Pedretti

Ielmini

2021

Electronics

View full text Add to dashboard Cite

show abstract

“…Due to the inherent stochastic nature of the ionic movement under Joule heating, the devices exhibit non-ideal characteristics, such as programming variability from cycle to cycle and from device to device, the asymmetry between the resistance increase (turn OFF – long term depression) and resistance decrease (turn ON – long term potentiation), read noise, and limited ON/OFF ratio or accessible resistance states. Other device indicators, such as device yield, read noise, and retention also impact their practical applicability ( Gokmen and Vlasov, 2016 ; Lin et al, 2019 ).…”

Section: Related Workmentioning

confidence: 99%

Gradient Decomposition Methods for Training Neural Networks With Non-ideal Synaptic Devices

et al. 2021

View full text Add to dashboard Cite

While promising for high-capacity machine learning accelerators, memristor devices have non-idealities that prevent software-equivalent accuracies when used for online training. This work uses a combination of Mini-Batch Gradient Descent (MBGD) to average gradients, stochastic rounding to avoid vanishing weight updates, and decomposition methods to keep the memory overhead low during mini-batch training. Since the weight update has to be transferred to the memristor matrices efficiently, we also investigate the impact of reconstructing the gradient matrixes both internally (rank-seq) and externally (rank-sum) to the memristor array. Our results show that streaming batch principal component analysis (streaming batch PCA) and non-negative matrix factorization (NMF) decomposition algorithms can achieve near MBGD accuracy in a memristor-based multi-layer perceptron trained on the MNIST (Modified National Institute of Standards and Technology) database with only 3 to 10 ranks at significant memory savings. Moreover, NMF rank-seq outperforms streaming batch PCA rank-seq at low-ranks making it more suitable for hardware implementation in future memristor-based accelerators.

show abstract

“…A potential issue for RRAM-based IMC is the limited precision of conductance, which is affected by programming variations [12], [13], random telegraph noise (RTN) [14], drift [15], and other types of random fluctuations [16], [17]. Additionally, reliability concerns at array-level such as conductance relaxation over time [18] and temperature instability [19] also challenge the achievement of a stable and high accuracy in RRAM-based IMC.…”

Section: Introductionmentioning

confidence: 99%

Accurate Program/Verify Schemes of Resistive Switching Memory (RRAM) for In-Memory Neural Network Circuits

Milo

Glukhov

Pérez

et al. 2021

IEEE Trans. Electron Devices

View full text Add to dashboard Cite

Resistive switching memory (RRAM) is a promising technology for embedded memory and their application in computing. In particular, RRAM arrays can provide a convenient primitive for matrix-vector multiplication (MVM) with strong impact on the acceleration of neural networks for artificial intelligence (AI). At the same time, RRAM is affected by intrinsic conductance variations which might cause a degradation of accuracy in AI inference hardware. This work provides a detailed study of the multilevel-cell (MLC) programming of RRAM for neural network applications. We compare three MLC programming schemes and discuss their variations in terms of the different slope in the programming characteristics. We test the accuracy of a 2layer fully-connected neural network (FC-NN) as a function of the MLC scheme, the number of weight levels, and the weight mapping configuration. We find a trade-off between the FC-NN accuracy, size and current consumption. This work highlights the importance of a holistic approach to AI accelerators encompassing the device properties, the overall circuit performance, and the AI application specifications. Index Terms-Resistive switching memory (RRAM); multilevel cell (MLC) operation; artificial neural network (ANN); in-memory computing (IMC).

show abstract

Performance Impacts of Analog ReRAM Non-ideality on Neuromorphic Computing

Cited by 61 publications

References 16 publications

In-Memory Computing with Resistive Memory Circuits: Status and Outlook

In-Memory Computing with Resistive Memory Circuits: Status and Outlook

Gradient Decomposition Methods for Training Neural Networks With Non-ideal Synaptic Devices

Accurate Program/Verify Schemes of Resistive Switching Memory (RRAM) for In-Memory Neural Network Circuits

Contact Info

Product

Resources

About