Spontaneous sparse learning for PCM-based memristor neural networks

Abstract

Neural networks trained by backpropagation have achieved tremendous successes on numerous intelligent tasks. However, naïve gradient-based training and updating methods on memristors impede applications due to intrinsic material properties. Here, we built a 39 nm 1 Gb phase change memory (PCM) memristor array and quantified the unique resistance drift effect. On this basis, spontaneous sparse learning (SSL) scheme that leverages the resistance drift to improve PCM-based memristor network training is developed. During training, SSL regards the drift effect as spontaneous consistency-based distillation process that reinforces the array weights at the high-resistance state continuously unless the gradient-based method switches them to low resistance. Experiments show that the SSL not only helps the convergence of network with better performance and sparsity controllability without additional computation in handwritten digit classification. This work promotes the learning algorithms with the intrinsic properties of memristor devices, opening a new direction for development of neuromorphic computing chips.

Memristive devices based hardware for unlabeled data processing

“…This strategy reduces the drop in inference accuracy significantly even under severe variations [156]. Nevertheless, some non-ideal effects in a proper range, such as intrinsic resistance drift of PCM [157], fluctuations and stochastic of RRAM [93] and intrinsic hardware noise of memristor [83] can be utilized to improve the accuracy of the computing system.…”

Effects of Thickness and Si3N4 Coverage on the Phase Transition Characteristics of GeTe Thin Films

“…Chalcogenide phase-change materials (PCMs) attract increasing interest for memory, memristor, and nonvolatile photonic applications due to excellent properties such as excellent scalability, low power, high speed, and nonvolatility, and the ability to crystallize gradually under electrical and optical excitation enables PCMs to realize multilevel states to extend their application for neurons and brain-inspired computing …”

Statistical Analysis of Uniform Switching Characteristics of Ta2O5-Based Memristors by Embedding In-Situ Grown 2D-MoS2 Buffer Layers

“…Extensive efforts have been conducted to realize an energy-efficient parallel computing architecture called a neuromorphic system, which would provide great potential applications, such as parallel data processing and unstructured pattern recognition [ 4 ]. Various types of non-volatile memory devices have been proposed for use in neuromorphic core hardware, such as resistive random-access memory [ 5 , 6 , 7 ] phase-change RAM [ 8 , 9 ] spin-transfer torque magnetoresistive RAM (STT-MRAM) [ 10 , 11 ], and conventional flash memory [ 12 ], respectively. Among them, RRAM devices are promising candidates for use in overcoming structural problems of von Neumann computing, because with these devices, computing and storage functions can be performed in the same circuit, enabling processing-in-memory (PIM) computing [ 13 , 14 , 15 , 16 , 17 ].…”