No abstract

Recent progress in novel non-volatile memory-based synaptic device technologies and their feasibility for matrix-vector multiplication (MVM) has ignited active research on implementing analog neural network training accelerators with resistive crosspoint arrays. While significant performance boost as well as area- and power-efficiency is theoretically predicted, the realization of such analog accelerators is largely limited by non-ideal switching characteristics of crosspoint elements. One of the most performance-limiting non-idealities is the conductance update asymmetry which is known to distort the actual weight change values away from the calculation by error back-propagation and, therefore, significantly deteriorates the neural network training performance. To address this issue by an algorithmic remedy, Tiki-Taka algorithm was proposed and shown to be effective for neural network training with asymmetric devices. However, a systematic analysis to reveal the required asymmetry specification to guarantee the neural network performance has been unexplored. Here, we quantitatively analyze the impact of update asymmetry on the neural network training performance when trained with Tiki-Taka algorithm by exploring the space of asymmetry and hyper-parameters and measuring the classification accuracy. We discover that the update asymmetry level of the auxiliary array affects the way the optimizer takes the importance of previous gradients, whereas that of main array affects the frequency of accepting those gradients. We propose a novel calibration method to find the optimal operating point in terms of device and network parameters. By searching over the hyper-parameter space of Tiki-Taka algorithm using interpolation and Gaussian filtering, we find the optimal hyper-parameters efficiently and reveal the optimal range of asymmetry, namely the asymmetry specification. Finally, we show that the analysis and calibration method be applicable to spiking neural networks.

Due to the rapid progress of artificial intelligence technology based on neural networks, the amount of required computation has been increasing dramatically. To keep up with the ever-increasing demand, novel analog neuromorphic computing architectures have been intensively studied, where cross-point arrays of resistive memory devices are utilized for high-speed and power-efficient computation. Among various synaptic memory device candidates, a metal oxide-based electrochemical random-access memory (MO-ECRAM) has been attractive due to its complementary metal-oxide-semiconductor-compatibility and superior programmability. In this work, we fabricate a WO 3 -based MO-ECRAM with multiple terminals and characterize the conductance modulation in the channel regions with and without the gate stack. While the gated region conductance shows a high on/off ratio, the ungated region conductance displays weak modulation with a near-unity on/off ratio. Based on our experimental observation, we propose a lithographical technique to intentionally uncover the channel area and utilize the ungated area's resistance to limit the maximum conductance of each cross-point element at the individual device level. We conduct a neural network training simulation for MNIST dataset and show that this technique can guarantee robust large array operations for high-performance neural network computation.

during data transfer between processor and memory. [6] To overcome this limitation, various research activities on computing in-memory have been conducted to optimize neural network computations. [7] Especially, numerous non-volatile memory devices have been proposed to update synaptic weights and perform matrix-vector multiplications (MVM). [8] However, most previous artificial synapse devices cannot satisfy all the requirements for high-performance neural network operations, for example, a large number of weight levels, wide conductance range, linear/symmetric weight update, outstanding device-to-device uniformity, low operating power, long retention, and good endurance. Resistive random access memory (RRAM) and phase change memory (PCM) exhibit a wide range of conductance changes, but show nonlinear weight modulation and poor device-todevice uniformity due to the abrupt transitions during device operation. [9][10][11] Spin-transfer torque magnetic random access memory shows extremely low programming delay and power consumption, but its conductance range is limited. [12] Flash memory and ferroelectric field-effect transistor (FeFET) exhibit outstanding CMOS compatibility and a wide conductance range; however, flash memory requires high operation voltage, and FeFET has poor fatigue characteristics, which results in cycle-tocycle variation. [13][14][15] To overcome the limitations of non-volatilememory-based synapse devices, metal-oxide-semiconductor This work presents an analog neuromorphic synapse device consisting of two oxide semiconductor transistors for high-precision neural networks. One of the two transistors controls the synaptic weight by charging or discharging the storage node, which leads to a conductance change in the other transistor. The programmed weight maintains for more than 300 s as electrons in the storage node are well preserved due to the extremely low off current of the oxide transistor. Ideal synaptic behaviors are achieved by utilizing superior properties of oxide transistors such as a high on/off ratio, low off current, and large-area uniformity. To further improve the synaptic performance, self-assembled monolayer treatment is applied for reducing the transistor conductance. The reduction of on current reduces the power consumption, and the reduced off current improves the retention characteristics. There is no noticeable decrease in simulated neural network accuracy even when the measured device-to-device variation is intentionally increased by 200%, indicating the possibility of large-array operation with the synapse device.

No abstract