Metal-oxide based, CMOS-compatible ECRAM for Deep Learning Accelerator

Kim, Seyoung; Ott, J. A.; Ando, Takashi; Miyazoe, Hiroyuki; Narayanan, V.; Rozen, John; Todorov, Teodor K.; Onen, Murat; Gokmen, Tayfun; Bishop, Douglas M.; Solomon, P. M.; Lee, Ko-Tao; Copel, M.; Farmer, Damon B.

doi:10.1109/iedm19573.2019.8993463

Cited by 77 publications

(103 citation statements)

References 4 publications

Supporting

Mentioning

100

Contrasting

Order By: Relevance

“…This ensures that any desired weight update can be realized with very high accuracy. The high accuracy of outer product updates demonstrated here is a vast improvement over past works (Fuller et al, 2019a;Kim et al, 2019) in both individual device switching behavior as well as device-to-device variation, and is essential for reaching convergence. As we show in simulation later, this quality strongly facilitates achieving rapid convergence in more complex problems involving hidden layers.…”

Section: Analysis Of Experimental Training Accuracymentioning

confidence: 85%

“…Three-terminal electrochemical random-access memory (ECRAM), also known as redox transistors, can address the accuracy, energy, and latency deficiencies of two-terminal memristors (Fuller et al, 2017(Fuller et al, , 2019bvan de Burgt et al, 2017;Sharbati et al, 2018;Tang et al, 2018;Kim et al, 2019;Li et al, 2019Li et al, , 2020aMelianas et al, 2020;Tuchman et al, 2020;Yao X. et al, 2020). ECRAM achieves exceptionally reproducible, linear, and symmetric weight updates by encoding information in resistance values that reflect changes in the average bulk concentration of dopants like protons in transistor-like channels.…”

Section: Introductionmentioning

confidence: 99%

“…While parallel weight updates have been shown using ECRAM (Fuller et al, 2019a;Kim et al, 2019), in situ training utilizing this outer product update has not been demonstrated due to overlapping fabrication and systems engineering challenges. Instead, numerical simulations (Agarwal et al, 2016;Jacobs-Gedrim et al, 2017;Bennett et al, 2019a;Kim et al, 2019) based on the switching properties of just one or few devices have been used to predict the accuracy of training large crossbar arrays (Fuller et al, 2017(Fuller et al, , 2019avan de Burgt et al, 2017;Li et al, 2019Li et al, , 2020a. Without experimental validation, it is unclear if such numerical simulations will accurately capture experimental training protocols, especially in the presence of non-ideal device behavior, variations between devices, loss of state, or sneak current pathways that are difficult to account for in array simulations using individual device measurements.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

In situ Parallel Training of Analog Neural Network Using Electrochemical Random-Access Memory

Xiao

Bennett

et al. 2021

Front. Neurosci.

View full text Add to dashboard Cite

In-memory computing based on non-volatile resistive memory can significantly improve the energy efficiency of artificial neural networks. However, accurate in situ training has been challenging due to the nonlinear and stochastic switching of the resistive memory elements. One promising analog memory is the electrochemical random-access memory (ECRAM), also known as the redox transistor. Its low write currents and linear switching properties across hundreds of analog states enable accurate and massively parallel updates of a full crossbar array, which yield rapid and energy-efficient training. While simulations predict that ECRAM based neural networks achieve high training accuracy at significantly higher energy efficiency than digital implementations, these predictions have not been experimentally achieved. In this work, we train a 3 × 3 array of ECRAM devices that learns to discriminate several elementary logic gates (AND, OR, NAND). We record the evolution of the network’s synaptic weights during parallel in situ (on-line) training, with outer product updates. Due to linear and reproducible device switching characteristics, our crossbar simulations not only accurately simulate the epochs to convergence, but also quantitatively capture the evolution of weights in individual devices. The implementation of the first in situ parallel training together with strong agreement with simulation results provides a significant advance toward developing ECRAM into larger crossbar arrays for artificial neural network accelerators, which could enable orders of magnitude improvements in energy efficiency of deep neural networks.

show abstract

Section: Analysis Of Experimental Training Accuracymentioning

confidence: 85%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

In situ Parallel Training of Analog Neural Network Using Electrochemical Random-Access Memory

Xiao

Bennett

et al. 2021

Front. Neurosci.

View full text Add to dashboard Cite

show abstract

“…Recently, a fully CMOS compatible ECRAM device was reported by exploiting fab-friendly oxygen anions and metal oxides as the mobile source and electrolyte/ channel, respectively, as shown in Figure 4b. [77] The ECRAM satisfied the requirements of the basic synaptic characteristics, and it was also experimentally demonstrated in small-sized arrays. [83] However, the achievable conductance range and operating conditions such as speed and voltage seemed to be strongly and sensitively correlated to the materials and geometry of each layer.…”

Section: Configurationmentioning

confidence: 83%

“…Other mobile cations such as H ion that emulates the role of the Li ions have also been examined, as shown in Table 2. [77,[79][80][81][82] Unlike the Li ions embedded in the host material, the gate voltage pushed the cations (e.g., H ion) toward the bottom of the electrolyte of WO 3 while attracting the electrons to the top of the channel. [82] It was discovered that the film quality and physical properties of each layer played a crucial role in determining the dynamic range of conductance.…”

Section: Configurationmentioning

confidence: 99%

Recent Advancements in Emerging Neuromorphic Device Technologies

Woo

Kim

et al. 2020

Advanced Intelligent Systems

View full text Add to dashboard Cite

The explosive growth of data and information has motivated technological developments in computing systems that utilize them for efficiently discovering patterns and gaining relevant insights. Inspired by the structure and functions of biological synapses and neurons in the brain, neural network algorithms that can realize highly parallel computations have been implemented on conventional silicon transistor‐based hardware. However, synapses composed of multiple transistors allow only binary information to be stored, and processing such digital states through complicated silicon neuron circuits makes low‐power and low‐latency computing difficult. Therefore, the attractiveness of the emerging memories and switches for synaptic and neuronal elements, respectively, in implementing neuromorphic systems, which are suitable for performing energy‐efficient cognitive functions and recognition, is discussed herein. Based on a literature survey, recent progress concerning memories shows that novel strategies related to materials and device engineering to mitigate challenges are presented to primarily achieve nonvolatile analog synaptic characteristics. Attempts to emulate the role of the neuron in various ways using compact switches and volatile memories are also discussed. It is hoped that this review will help direct future interdisciplinary research on device, circuit, and architecture levels of neuromorphic systems.

show abstract