Experimental Demonstration of Supervised Learning in Spiking Neural Networks with Phase-Change Memory Synapses

Abstract

Spiking neural networks (Snn) are computational models inspired by the brain's ability to naturally encode and process information in the time domain. the added temporal dimension is believed to render them more computationally efficient than the conventional artificial neural networks, though their full computational capabilities are yet to be explored. Recently, in-memory computing architectures based on non-volatile memory crossbar arrays have shown great promise to implement parallel computations in artificial and spiking neural networks. In this work, we evaluate the feasibility to realize high-performance event-driven in-situ supervised learning systems using nanoscale and stochastic analog memory synapses. For the first time, the potential of analog memory synapses to generate precisely timed spikes in Snns is experimentally demonstrated. the experiment targets applications which directly integrates spike encoded signals generated from bio-mimetic sensors with in-memory computing based learning systems to generate precisely timed control signal spikes for neuromorphic actuators. More than 170,000 phase-change memory (PCM) based synapses from our prototype chip were trained based on an event-driven learning rule, to generate spike patterns with more than 85% of the spikes within a 25 ms tolerance interval in a 1250 ms long spike pattern. We observe that the accuracy is mainly limited by the imprecision related to device programming and temporal drift of conductance values. We show that an array level scaling scheme can significantly improve the retention of the trained Snn states in the presence of conductance drift in the pcM. combining the computational potential of supervised Snns with the parallel compute power of inmemory computing, this work paves the way for next-generation of efficient brain-inspired systems.

Phase Change Random Access Memory for Neuro‐Inspired Computing

“…Many significant applications in the field of neuro‐inspired computing, particularly in image classification and identification, have been successfully implemented by SNN supervised learning. [ 60,67,85–89 ]…”

“…[ 92 ] d) SNN. [ 93 ] e) The training of supervised learning SNN. The audio signal of characters “IBM” (Eye..Bee..Em) was captured and then converted into 132 spike streams that were encoded in designed SNN.…”

Memristors—From In‐Memory Computing, Deep Learning Acceleration, and Spiking Neural Networks to the Future of Neuromorphic and Bio‐Inspired Computing

“…[88] In the experiment, 132 spike streams representing spoken audio signals generated using a silicon cochlea chip were used as the input, and the network was trained to generate 168 spike streams whose arrival times indicate the pixel intensity corresponding to the spoken characters. [88] Compared with normal classification problems in deep networks where the accuracy depends only on the relative magnitude of the response of the output neurons, the SNN problem is harder, as the network is tasked with generating close to 1000 spikes at specific time instances over a period of 1250 ms from 168 spiking neurons that are excited by 132 input spike streams. The accuracy for spike placement obtained in the experiment was about 80% compared with the software baseline accuracy of over 98%, despite using the same multi-memristive architecture described earlier.…”

“…Recently, Nandakumar et al demonstrated a proof‐of‐concept realization of supervised learning in a two‐layer SNN implemented using nanoscale PCM synapses based on the Normalized Approximate Descent Algorithm. [ 88 ] In the experiment, 132 spike streams representing spoken audio signals generated using a silicon cochlea chip were used as the input, and the network was trained to generate 168 spike streams whose arrival times indicate the pixel intensity corresponding to the spoken characters. [ 88 ] Compared with normal classification problems in deep networks where the accuracy depends only on the relative magnitude of the response of the output neurons, the SNN problem is harder, as the network is tasked with generating close to 1000 spikes at specific time instances over a period of 1250 ms from 168 spiking neurons that are excited by 132 input spike streams.…”

Novel hardware and concepts for unconventional computing

“…In this respect, Araujo et al 21 discuss the role of non-linear data processing on a speech recognition task, while Chou et al 22 report on an analogue computing system with coupled non-linear oscillators which is capable of solving complex combinatorial optimization problems. Nandakumar et al 23 demonstrate experimentally a supervised learning scheme using spiking neurons and phase change materials, while in 24,25 photonic based-neuromorphic computing schemes are presented. For a significant reduction in power consumption, wake-up systems are of interest, which only switch on the more complex computing structures when they are needed.…”