Simulation of Inference Accuracy Using Realistic RRAM Devices

Abstract

Resistive Random Access Memory (RRAM) is a promising technology for power efficient hardware in applications of artificial intelligence (AI) and machine learning (ML) implemented in non-von Neumann architectures. However, there is an unanswered question if the device non-idealities preclude the use of RRAM devices in this potentially disruptive technology. Here we investigate the question for the case of inference. Using experimental results from silicon oxide (SiO x ) RRAM devices, that we use as proxies for physical weights, we demonstrate that acceptable accuracies in classification of handwritten digits (MNIST data set) can be achieved using non-ideal devices. We find that, for this test, the ratio of the high- and low-resistance device states is a crucial determinant of classification accuracy, with ~96.8% accuracy achievable for ratios >3, compared to ~97.3% accuracy achieved with ideal weights. Further, we investigate the effects of a finite number of discrete resistance states, sub-100% device yield, devices stuck at one of the resistance states, current/voltage non-linearities, programming non-linearities and device-to-device variability. Detailed analysis of the effects of the non-idealities will better inform the need for the optimization of particular device properties.

Complementary Metal‐Oxide Semiconductor and Memristive Hardware for Neuromorphic Computing

“…Some of these devices also present a multilevel structure, which is useful in implementing neuromorphic computing systems. It is worth noting that, not all designs, except for two of our previous works are able to implement the three investigated synaptic plasticity functionalities, i.e., LTP and LTD, and STDP.…”

“…SiO x memristive devices are used to implement both STDP and synaptic weights in physical implementations of artificial neural networks . STDP is often regarded as a key local learning rule in biological systems, modulating synaptic weights in accordance with the degree of temporal overlap between pre‐ and postsynaptic action potentials.…”

“…These include device‐to‐device and cycle‐to‐cycle variabilities, a limited number of resistance states or a narrow operational range of resistance modulation, nonlinearity of both voltage–current characteristics and pulse response of devices, and high line resistances in crossbar arrays. Despite these, SiO x ‐RRAM devices have been used to implement weights in physical neural networks, in which multiple weight values (memristive resistance states) are required …”

“…e) Three very different LTD curves are obtained from the same RRAM device by changing the pulse amplitude and width. Reproduced under the terms of the CC BY license . Copyright 2019.…”

“…We have studied extensively the effects of various memristors’ nonidealities on inference accuracy, using the MNIST handwritten digits dataset, where the neural network weights are represented as memristive device‐resistant states, as shown in Figure . Figure shows examples of the impact of yield and device failure on inference accuracy, as well as that of varying the number of resistance states with different schemes to map device resistance onto ANN weights, as shown in Figure .…”