In situ Parallel Training of Analog Neural Network Using Electrochemical Random-Access Memory

Abstract

In-memory computing based on non-volatile resistive memory can significantly improve the energy efficiency of artificial neural networks. However, accurate in situ training has been challenging due to the nonlinear and stochastic switching of the resistive memory elements. One promising analog memory is the electrochemical random-access memory (ECRAM), also known as the redox transistor. Its low write currents and linear switching properties across hundreds of analog states enable accurate and massively parallel updates of a full crossbar array, which yield rapid and energy-efficient training. While simulations predict that ECRAM based neural networks achieve high training accuracy at significantly higher energy efficiency than digital implementations, these predictions have not been experimentally achieved. In this work, we train a 3 × 3 array of ECRAM devices that learns to discriminate several elementary logic gates (AND, OR, NAND). We record the evolution of the network’s synaptic weights during parallel in situ (on-line) training, with outer product updates. Due to linear and reproducible device switching characteristics, our crossbar simulations not only accurately simulate the epochs to convergence, but also quantitatively capture the evolution of weights in individual devices. The implementation of the first in situ parallel training together with strong agreement with simulation results provides a significant advance toward developing ECRAM into larger crossbar arrays for artificial neural network accelerators, which could enable orders of magnitude improvements in energy efficiency of deep neural networks.

Hafnium Oxide (HfO2) – A Multifunctional Oxide: A Review on the Prospect and Challenges of Hafnium Oxide in Resistive Switching and Ferroelectric Memories

“…Several groups have explored metrology parameters along the device and system level modeling approach to understand the design and optimization of switching devices and arrays. [217][218][219][220][221][222][223][224][225][226] In the field of NVM's simulation tools can be used to understand the physical design parameters, experimental data, process optimizations, performance prediction, and so on. However a single model can't fulfill all the requirements.…”

“…[1][2][3][4][5][6][7][8][9] Among several adopted high-k materials, sub-stoichiometric hafnium oxide (HfO x ) or switching properties and predicted similar behavior. [217][218][219][220][221][222][223][224][225][226] Both theoretical and experimental findings equip this technology for emerging applications. [227][228][229][230][231][232][233][234][235][236][237][238][239][240][241][242][243][244] Depending on the structural design and materials, the switching in RRAM devices can be further classified as volatile threshold switch (TS) and non-volatile memory switch (MS).…”

“…[ 151,154,214–216 ] Previously, several groups modeled the switching properties and predicted similar behavior. [ 217–226 ] Both theoretical and experimental findings equip this technology for emerging applications. [ 227–244 ] Depending on the structural design and materials, the switching in RRAM devices can be further classified as volatile threshold switch (TS) and non‐volatile memory switch (MS).…”

High‐Performance Organic Electrochemical Transistors and Neuromorphic Devices Comprising Naphthalenediimide‐Dialkoxybithiazole Copolymers Bearing Glycol Ether Pendant Groups

“…As illustrated in Figure 5a, a series resistor is used to control the gate current and prevent the loss of the conductance state. [ 49 ] A voltage pulse (amplitude +10 V for potentiation and –3 V for depression, width of 300 ms) is applied at the gate. The inset is a zoom‐in image clearly showing the individual conductance states during 500 ms.…”

Metaplastic and energy-efficient biocompatible graphene artificial synaptic transistors for enhanced accuracy neuromorphic computing

“…Coupled with the slow kinetics of the proton cluster movement within the dry Nafion matrix, long-term potentiation can be explained. It should be noted that voltage pulses can also be used in place of current pulses with the addition of a switch at the gate to open the circuit when a pulse is not being delivered, a common technique to enhance state retention in electrochemical devices 8,45 . In the case of the BLASTs, current pulses are used due to ease of use in the experimental setup and to promote the linearity and symmetry of synaptic response because the amount of charge delivered to the device mediates the change in conductance.…”