MemTorch: An Open-source Simulation Framework for Memristive Deep Learning Systems

Lammie, Corey; Wang, Xiang; Linares-Barranco, B.; Azghadi, Mostafa Rahimi

doi:10.1016/j.neucom.2022.02.043

Cited by 41 publications

(20 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is mainly due to unavailability of experimental data, which resulted in developing an empirical, rather than a physics-based model. Additionally, while this work only focuses on endurance and retention and their impact on memristive deep learning networks performance, future improvements of our model can account for modelling a nite number of conductance states and other device nonidealities [32,36].…”

Section: Discussionmentioning

confidence: 99%

“…Vstop dependence is validated in log(Cycles to Failure) architectures contain several modular crossbar tiles connected using a shared bus. (D) Modular crossbar tiles consist of crossbar arrays with supporting peripheral circuitry, and can represent weights using a dual-array scheme (as depicted), a dual row scheme, where double the number of rows are required, or a current-mirror scheme, that is capable of operation using a singular device to represent each weight [32].…”

Section: Model Validationmentioning

confidence: 99%

See 1 more Smart Citation

Empirical metal-oxide RRAM device endurance and retention model for deep learning simulations

Lammie

Azghadi

Ielmini

2021

Semicond. Sci. Technol.

Self Cite

View full text Add to dashboard Cite

Memristive devices including resistive random access memory (RRAM) cells are promising nanoscale low-power components projected to facilitate significant improvement in power and speed of Deep Learning (DL) accelerators, if structured in crossbar architectures. However, these devices possess non-ideal endurance and retention properties, which should be modeled efficiently. In this paper, we propose a novel generalized empirical metal-oxide RRAM endurance and retention model for use in large-scale DL simulations. To the best of our knowledge, the proposed model is the first to unify retention-endurance modeling while taking into account time, energy, SET-RESET cycles, device size, and temperature. We compare the model to state-of-the-art and demonstrate its versatility by applying it to experimental data from fabricated devices. Furthermore, we use the model for CIFAR-10 dataset classification using a large-scale deep memristive neural network (DMNN) implementing the MobileNetV2 architecture. Our results show that, even when ignoring other device non-idealities, retention and endurance losses significantly affect the performance of DL networks. Our proposed model and its DL simulations are made publicly available.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Model Validationmentioning

confidence: 99%

Empirical metal-oxide RRAM device endurance and retention model for deep learning simulations

Lammie

Azghadi

Ielmini

2021

Semicond. Sci. Technol.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Memristive devices can be arranged within crossbar architectures to perform Vector Matrix Multiplications (VMMs) in-memory, in Op1q [14], which are used extensively in forward and backward propagations within Convolutional Neural Networks (CNNs) to compute the output of fully connected and unrolled convolutional layers. Scaled weight matrices can either be represented using two crossbars per layer, g pos and g neg , to represent positive and negative weights, respectively, or using a singular crossbar per layer with current mirrors, so that the effective conductance of each device is offset by a fixed value, g m , that can be determined using (1) [15]…”

Section: B Memristive DL Systemsmentioning

confidence: 99%

“…The MemTorch [15] simulation framework was used to simulate RRAM devices during inference using the VTEAM [21] model. Performance metrics for our trained conventional and equivalent MDLS are reported in Table II.…”

Section: Performance Evaluationmentioning

confidence: 99%

“…Two non-ideal device characteristics were modelled: device-todevice variability, and a finite number of discrete conductance states. Device-device variability was introduced stochastically by sampling R ON and R OFF for each device from a normal distribution with R ON " 100Ω and σ, and R OFF " 2, 500 and 2σ, as R OFF " R ON [15], for σ = 0-500. As it has been demonstrated that the spacing between states is not critical [23], we simulated devices with between 2-10 uniformly distributed conductance states.…”

Section: Performance Evaluationmentioning

confidence: 99%

See 1 more Smart Citation

Towards Memristive Deep Learning Systems for Real-time Mobile Epileptic Seizure Prediction

Lammie,

Xiang,

Azghadi

2021

Preprint

Self Cite

View full text Add to dashboard Cite

The unpredictability of seizures continues to distress many people with drug-resistant epilepsy. On account of recent technological advances, considerable efforts have been made using different hardware technologies to realize smart devices for the real-time detection and prediction of seizures. In this paper, we investigate the feasibility of using Memristive Deep Learning Systems (MDLSs) to perform real-time epileptic seizure prediction on the edge. Using the MemTorch simulation framework and the Children's Hospital Boston (CHB)-Massachusetts Institute of Technology (MIT) dataset we determine the performance of various simulated MDLS configurations. An average sensitivity of 77.4% and a Area Under the Receiver Operating Characteristic Curve (AUROC) of 0.85 are reported for the optimal configuration that can process Electroencephalogram (EEG) spectrograms with 7,680 samples in 1.408ms while consuming 0.0133W and occupying an area of 0.1269mm 2 in a 65nm Complementary Metal-Oxide-Semiconductor (CMOS) process.

show abstract

Scalable Synaptic Transistor Memory from Solution‐Processed Carbon Nanotubes for High‐Speed Neuromorphic Data Processing

Pei,

Song,

Liu

et al. 2024

Advanced Materials

View full text Add to dashboard Cite

Neural networks as a core information processing technology in machine learning and artificial intelligence demand substantial computational resources to deal with the extensive multiply‐accumulate operations. Neuromorphic computing is an emergent solution to address this problem, allowing the computation performed in memory arrays in parallel with high efficiencies conforming to the neural networks. Here, scalable synaptic transistor memories are developed from solution‐sorted carbon nanotubes. The transistors exhibit a large switching ratio of over 105, a significant memory window of ≈12 V arising from charge trapping, and low response delays down to tens of nanoseconds. These device characteristics endow highly stabilized reconfigurable conductance states, successful emulation of synaptic functions, and a high data processing speed. Importantly, the devices exhibit uniform characteristic metrics, e.g., with a 1.8% variation in the memory window, suggesting an industrial‐scale manufacturing capability of the fabrication. Using the memories, a hardware convolution kernel is designed and parallel image processing is demonstrated at a speed of 1 M bit per second per input channel. Given the efficacy of the convolution kernel, a promising prospect of the memories in implementing neuromorphic computing is envisaged. To explore the potential, large‐scale convolution kernels are simulated and high‐speed video processing is realized for autonomous driving.

show abstract

MemTorch: An Open-source Simulation Framework for Memristive Deep Learning Systems

Cited by 41 publications

References 47 publications

Empirical metal-oxide RRAM device endurance and retention model for deep learning simulations

Empirical metal-oxide RRAM device endurance and retention model for deep learning simulations

Towards Memristive Deep Learning Systems for Real-time Mobile Epileptic Seizure Prediction

Scalable Synaptic Transistor Memory from Solution‐Processed Carbon Nanotubes for High‐Speed Neuromorphic Data Processing

Contact Info

Product

Resources

About