RRAM for Compute-in-Memory: From Inference to Training

Yu, Shimeng; Shim, Wonbo; Peng, Xiaochen; Luo, Yandong

doi:10.1109/tcsi.2021.3072200

Cited by 73 publications

(22 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, it has been reported that to avoid the large decline in inference accuracy of the CIFAR-10 dataset, less than 1% variation in the overall conductivity range of RRAM is required. [22] Favorably, cycle-tocycle variation is mainly attenuated through iterative training and can be improved through the write-verify process. [23,24] In addition, small noise injected during training can improve the robustness for subsequent inference variation by preventing convergence to the local minima in the energy landscape of the DNN model.…”

Section: Variation Device Sizementioning

confidence: 99%

Recent Advances in Synaptic Nonvolatile Memory Devices and Compensating Architectural and Algorithmic Methods Toward Fully Integrated Neuromorphic Chips

Byun

Choi

Kwon

et al. 2022

Adv Materials Technologies

View full text Add to dashboard Cite

Nonvolatile memory (NVM)‐based neuromorphic computing has been attracting considerable attention from academia and the industry. Although it is not completely successful yet, remarkable achievements have been reported pertaining to synaptic devices that can leverage NVM capable of storing multiple states. The analog synaptic devices performing computation similar to biological nerve systems are crucial in energy‐efficient analog neuromorphic computing systems. To use NVM as an analog synaptic device, researchers focus on improving device characteristics. Among various characteristics, the most challenging one is linearity and symmetry of synaptic weight update that is required for on‐chip training. In this regard, this review paper discusses recent synaptic device improvements focusing on novel schemes tailored for each NVM device to improve the linearity and symmetry. In addition to device‐level studies, recent research achievements are reviewed expanded up to chip‐level studies because in realizing neuromorphic hardware systems beyond a single synaptic device, several considerations and requirements are needed to confirm for high‐level design, and accordingly, cooptimize among synaptic devices, synapse arrays, electrical circuits, neural networks, algorithms, and implementation. Also, this review paper introduces various circuit and algorithmic approaches to compensate for the non‐ideality of the analog synaptic device.

show abstract

Section: Variation Device Sizementioning

confidence: 99%

Recent Advances in Synaptic Nonvolatile Memory Devices and Compensating Architectural and Algorithmic Methods Toward Fully Integrated Neuromorphic Chips

Byun

Choi

Kwon

et al. 2022

Adv Materials Technologies

View full text Add to dashboard Cite

show abstract

“…where G ij is the programmed conductance value, representing the network weight. Since it is not possible to program memristor cells to negative conductances, two adjacent crossbars or bit-lines are used in tandem instead [11], such that the actual weight value is G crossbar1 − G crossbar2 , this also allows us to double the effective range of an n-bit quantization scheme without adding a complement bit, since both negative and positive weights can be represented in nbits just on different crossbars, we will be utilising this fact throughout this work.…”

Section: Preliminaries a Memristor Crossbars As Neural Network Accele...mentioning

confidence: 99%

PRUNIX: Non-Ideality Aware Convolutional Neural Network Pruning for Memristive Accelerators

Alshaarawy¹,

Amirsoleimani²,

Genov³

2022

Preprint

View full text Add to dashboard Cite

In this work, PRUNIX, a framework for training and pruning convolutional neural networks is proposed for deployment on memristor crossbar based accelerators. PRUNIX takes into account the numerous non-ideal effects of memristor crossbars including weight quantization, state-drift, aging and stuck-at-faults. PRUNIX utilises a novel Group Sawtooth Regularization intended to improve non-ideality tolerance as well as sparsity, and a novel Adaptive Pruning Algorithm (APA) intended to minimise accuracy loss by considering the sensitivity of different layers of a CNN to pruning. We compare our regularization and pruning methods with other standards on multiple CNN architectures, and observe an improvement of 13% test accuracy when quantization and other non-ideal effects are accounted for with an overall sparsity of 85%, which is similar to other methods.

show abstract

“…The study conducted by Yu et al in Ref. [31] using TSMC ® 40nm RRAM technology and Intel ® 22nm RRAM technology was used to build a VGG-8 NN and trained on a CIFAR-10 dataset using a modified Neurosim, with the intention of optimizing the use of ADC by using a MUX based ADC. The same group utilized a mixed RRAM design with RRAM memory designed for MSB bits, and a regular memory used for LSB bits as shown in Ref.…”

Section: B Variability Study In Neuromorphic Learning Circuits On The...mentioning

confidence: 99%

Neuromorphic In-Memory RRAM NAND/NOR Circuit Performance Analysis in a CNN Training Framework on the Edge for Low Power IoT

Prabhu

Raghavan

2022

IEEE Access

View full text Add to dashboard Cite

Training a CNN involves computationally intense optimization algorithms to fit the network using a training dataset, to update the network weight for inferencing and then pattern classification. Hence, the application of in-memory computation would enable a highly power-efficient low latency on-the-edge CNN training technique by avoiding the memorywall created during the external memory read/write operation (for off chip instruction and data transfer). A memory writeverify, and re-program technique can control the RRAM variability. Still, memory verification and re-program is a complex process with additional resources needed for practical implementation of verification circuit. In this study, we have demonstrated a practical (First-in Max-Out) FIMO-based cache memory called Maximum Count Binary Comparator Layer (MCBC), using 1T3R, 1T5R, and 1T7R RRAM structures by using a probability-based accuracy improvement architecture, without the conventional verification process. We constructed 10 layered modified MobileNET with filter size ranging from 32 -512 and trained with Traffic Sign Recognition Database (TSRD) using a three-tier abstraction simulation learning framework -(1) High level, 10 layered CNN implementation with Python+TensorFlow; (2) Verilog HDL based FP32MUL and FP32ADD (32-bits Floating Point adder and multiplier) circuits constructed with RRAM NAND gates using 1T2R structures; and (3) Digital Look-Up-Table (LUT) model for RRAM variability. An edge learning framework (for the forward pass) is demonstrated using digital RRAM-NAND/NOR universal gates integrated with the Maximum Count Binary Comparator Layer (MCBC) to partially circumvent the impact of RRAM variability and to quantify the RRAM variability on the CNN training prediction accuracy for 65nm CMOS OxRAM (TiN/HfO2/Hf/TiN) with varying device current compliance of 5, 10, and 50µA for low power IoT applications. The MCBC layer was simulated using a SPICE model, for which the estimated chip layout is 1150 × 1230 nm 2 per logical gate input, which resulted in an overall prediction accuracy improvement from 10% to 60% by repeating the logical operations of the NOR gate for {1, 3, 5, and 7} cycles respectively.

show abstract

RRAM for Compute-in-Memory: From Inference to Training

Cited by 73 publications

References 57 publications

Recent Advances in Synaptic Nonvolatile Memory Devices and Compensating Architectural and Algorithmic Methods Toward Fully Integrated Neuromorphic Chips

Recent Advances in Synaptic Nonvolatile Memory Devices and Compensating Architectural and Algorithmic Methods Toward Fully Integrated Neuromorphic Chips

PRUNIX: Non-Ideality Aware Convolutional Neural Network Pruning for Memristive Accelerators

Neuromorphic In-Memory RRAM NAND/NOR Circuit Performance Analysis in a CNN Training Framework on the Edge for Low Power IoT

Contact Info

Product

Resources

About