Learning Real-World Stimuli in a Neural Network with Spike-Driven Synaptic Dynamics

Abstract

We present a model of spike-driven synaptic plasticity inspired by experimental observations and motivated by the desire to build an electronic hardware device that can learn to classify complex stimuli in a semisupervised fashion. During training, patterns of activity are sequentially imposed on the input neurons, and an additional instructor signal drives the output neurons toward the desired activity. The network is made of integrate-and-fire neurons with constant leak and a floor. The synapses are bistable, and they are modified by the arrival of presynaptic spikes. The sign of the change is determined by both the depolarization and the state of a variable that integrates the postsynaptic action potentials. Following the training phase, the instructor signal is removed, and the output neurons are driven purely by the activity of the input neurons weighted by the plastic synapses. In the absence of stimulation, the synapses preserve their internal state indefinitely. Memories are also very robust to the disruptive action of spontaneous activity. A network of 2000 input neurons is shown to be able to classify correctly a large number (thousands) of highly overlapping patterns (300 classes of preprocessed Latex characters, 30 patterns per class, and a subset of the NIST characters data set) and to generalize with performances that are better than or comparable to those of artificial neural networks. Finally we show that the synaptic dynamics is compatible with many of the experimental observations on the induction of long-term modifications (spike-timing-dependent plasticity and its dependence on both the postsynaptic depolarization and the frequency of pre- and postsynaptic neurons).

Connectivity reflects coding: a model of voltage-based STDP with homeostasis

“…Deriving STDP rules from voltage dependence has been attempted before 18,41--42 . However, since these earlier models use the momentary voltage 42 or its derivative 41 , rather than a combination of momentary and averaged voltage as in our model, these earlier models cannot account for the broad range of nonlinear effects in STDP experiments or interaction of voltage and spike--timing. The voltage--based model of Sjöström 18 uses separate empirical functions for timing dependence, voltage dependence, frequency dependence, and multiple spike summation with preference for LTP, to capture the nonlinear effects of LTP.…”

Evaluating SPAN Incremental Learning for Handwritten Digit Recognition

“…After training, the synaptic weights take positive (excitatory) and negative (inhibitory) values This means the neuron is learning its class and also learning to reject other classes through weights adjustment. It might be more desirable to design a mechanism that is based on inhibition to suppress the neuron firing, for example using a similar mechanism to "winner-takes-all" as it is in [1]. In addition, the used network consists of a single layer of spiking neurons and there are no specific features extracted for classification.…”

Evolving Spiking Neural Networks and Neurogenetic Systems for Spatio- and Spectro-Temporal Data Modelling and Pattern Recognition

“…Temporal spike learning is observed in the auditory [93], the visual [11] and the motor control information processing of the brain [13,90]. Its use in neuro-prosthetics is essential, along with applications for a fast, real-time recognition and control of sequence of related processes [14].…”

“…Temporal coding accounts for the precise time of spikes and has been utilised in several learning rules, most popular being Spike-Time Dependent Plasticity (STDP) [103,69] and SDSP [30,14]. Temporal coding of information in SNN makes use of the exact time of spikes (e.g.…”

“…The SDSP is an unsupervised learning method [30,14], a modification of the STDP, that directs the change of the synaptic plasticity V w0 of a synapse w 0 depending on the time of spiking of the pre-synaptic neuron and the post-synaptic neuron. V w0 increases or decreases, depending on the relative timing of the pre-and post-synaptic spikes.…”