Pathways to efficient neuromorphic computing with non-volatile memory technologies

Abstract

Historically, memory technologies have been evaluated based on their storage density, cost, and latencies. Beyond these metrics, the need to enable smarter and intelligent computing platforms at a low area and energy cost has brought forth interesting avenues for exploiting non-volatile memory (NVM) technologies. In this paper, we focus on non-volatile memory technologies and their applications to bio-inspired neuromorphic computing, enabling spike-based machine intelligence. Spiking neural networks (SNNs) based on discrete neuronal “action potentials” are not only bio-fidel but also an attractive candidate to achieve energy-efficiency, as compared to state-of-the-art continuous-valued neural networks. NVMs offer promise for implementing both area- and energy-efficient SNN compute fabrics at almost all levels of hierarchy including devices, circuits, architecture, and algorithms. The intrinsic device physics of NVMs can be leveraged to emulate dynamics of individual neurons and synapses. These devices can be connected in a dense crossbar-like circuit, enabling in-memory, highly parallel dot-product computations required for neural networks. Architecturally, such crossbars can be connected in a distributed manner, bringing in additional system-level parallelism, a radical departure from the conventional von-Neumann architecture. Finally, cross-layer optimization across underlying NVM based hardware and learning algorithms can be exploited for resilience in learning and mitigating hardware inaccuracies. The manuscript starts by introducing both neuromorphic computing requirements and non-volatile memory technologies. Subsequently, we not only provide a review of key works but also carefully scrutinize the challenges and opportunities with respect to various NVM technologies at different levels of abstraction from devices-to-circuit-to-architecture and co-design of hardware and algorithm.

Density functional theory insight into metal ions and vacancies for improved performance in storage devices

“…Inherently low voltage (<2 V) 11 and power consumption of RRAM leverage them in the industry of energy saving devices 2,12‐14 . The review of Chakraborty et al was focused on low energy aspect of NVM and their application in energy efficient memory and computing devices 15 . Our research group has reported earlier various characteristics of low power RRAM devices in different RS materials 14,16,17 .…”

Short-Term Memory Dynamics of TiN/Ti/TiO2/SiOx/Si Resistive Random Access Memory

“…As shown in numerous previous studies, RRAM has been extensively reported to mimic synaptic characteristics [ 29 , 30 , 31 , 32 , 33 ] as well as to implement nonvolatile high-density memory [ 34 ]. Anion migration and metal ion migration are representative RRAM operation systems [ 32 ].…”

Multi-Level Neuromorphic Devices Built on Emerging Ferroic Materials: A Review

“…As for filament-based ReRAM with oxides such as HfO x /TaO x , although it has advantages of compact cell/array size, large device variability (especially at high resistance states) can be a major hindrance (Yu and Chen, 2016 ; Li et al, 2019 ). The large device variation not only places challenges on the sensing circuit but also leads to a reduced number of bits per cell, even when the device-level conductance ON/OFF ratio is high (Chakraborty et al, 2020b ). On the other hand, the memory effect of ferroic (ferromagnetic or ferroelectric) orderings are well-poised given their advantages in storing information with superior retention.…”

“…Moreover, due to the large conductance drift over time in PCM, erroneous results may occur even for inference-only tasks when running a pre-trained model mapped in PCM crossbar arrays. At present, although various types of NVM devices have been proposed and studied, it is still challenging to provide a reliable, scalable, and energy efficient hardware solution for multi-level neuro-mimetic devices (Burr et al, 2017 ; Schuman et al, 2017 ; Yan et al, 2018 ; Chakraborty et al, 2020b ; Kim et al, 2020 ).…”