A spiking network that learns to extract spike signatures from speech signals

Tavanaei, Amirhossein; Maida, Anthony S.

doi:10.1016/j.neucom.2017.01.088

Cited by 48 publications

(23 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this approach, firing the target neuron causes STDP over the incoming synapses and firing the non-target neurons causes anti-STDP. This approach has successfully been used in SNNs for numerical data classification [120], handwritten digit recognition [121], spoken digit classification [83], and reinforcement learning in SNNs [122]. The sharp synaptic weight adaptation based on immediate STDP and anti-STDP results in fast learning.…”

Section: B Learning Rules In Snnsmentioning

confidence: 99%

Deep learning in spiking neural networks

et al. 2019

Self Cite

View full text Add to dashboard Cite

1 In recent years, deep learning has revolutionized the field of machine learning, for computer vision in particular. In this approach, a deep (multilayer) artificial neural network (ANN) is trained in a supervised manner using backpropagation. Vast amounts of labeled training examples are required, but the resulting classification accuracy is truly impressive, sometimes outperforming humans. Neurons in an ANN are characterized by a single, static, continuous-valued activation. Yet biological neurons use discrete spikes to compute and transmit information, and the spike times, in addition to the spike rates, matter. Spiking neural networks (SNNs) are thus more biologically realistic than ANNs, and arguably the only viable option if one wants to understand how the brain computes. SNNs are also more hardware friendly and energy-efficient than ANNs, and are thus appealing for technology, especially for portable devices. However, training deep SNNs remains a challenge. Spiking neurons' transfer function is usually non-differentiable, which prevents using backpropagation.Here we review recent supervised and unsupervised methods to train deep SNNs, and compare them in terms of accuracy, but also computational cost and hardware friendliness. The emerging picture is that SNNs still lag behind ANNs in terms of accuracy, but the gap is decreasing, and can even vanish on some tasks, while SNNs typically require many fewer operations.

show abstract

Section: B Learning Rules In Snnsmentioning

confidence: 99%

Deep learning in spiking neural networks

et al. 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…For a system matrix A of size N ×N in a discrete state space system which defines the time dynamics, we get its diagonal entries in vector a. From this, we propose the memory metric τ M for an approximate model of the reservoir (13) to be (22). In our case, discrete time step h is 1ms.…”

Section: B Concept Of Memorymentioning

confidence: 99%

“…Performance of the LSM for various synaptic weight scaling is found ( Fig. 4 (b)) and the memory metric τ M of the corresponding state space modelled system is found using (22), and its variation against synaptic weight scaling is shown in Fig. 4 (c).…”

Section: B Performance As a Function Of Network Activitymentioning

confidence: 99%

Predicting Performance using Approximate State Space Model for Liquid State Machines

Gorad

Saraswat

Ganguly

2019

2019 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

Liquid State Machine (LSM) is a brain-inspired architecture used for solving problems like speech recognition and time series prediction. LSM comprises of a randomly connected recurrent network of spiking neurons. This network propagates the non-linear neuronal and synaptic dynamics. Maass et al. have argued that the non-linear dynamics of LSMs is essential for its performance as a universal computer. Lyapunov exponent (µ), used to characterize the non-linearity of the network, correlates well with LSM performance. We propose a complementary approach of approximating the LSM dynamics with a linear state space representation. The spike rates from this model are well correlated to the spike rates from LSM. Such equivalence allows the extraction of a memory metric (τM ) from the state transition matrix. τM displays high correlation with performance. Further, high τM system require lesser epochs to achieve a given accuracy. Being computationally cheap (1800× time efficient compared to LSM), the τM metric enables exploration of the vast parameter design space. We observe that the performance correlation of the τM surpasses the Lyapunov exponent (µ), (2 − 4× improvement) in the high-performance regime over multiple datasets. In fact, while µ increases monotonically with network activity, the performance reaches a maxima at a specific activity described in literature as the edge of chaos. On the other hand, τM remains correlated with LSM performance even as µ increases monotonically. Hence, τM captures the useful memory of network activity that enables LSM performance. It also enables rapid design space exploration and fine-tuning of LSM parameters for high performance.

show abstract

“…These models present single layer supervised learning. However, the idea of spike/no-spike classification broadens a new supervised learning category incorporating efficient spike-timing-dependent plasticity (STDP) [28,29,30] and anti-STDP that are triggered according to the neuron's label [31,32]. STDP is a biologically plausible learning rule occurs in the brain in which the presynaptic spikes occur immediately before the current postsynaptic spike strengthen the interconnecting synapses (LTP); otherwise, the synapses are weakened (LTD).…”

Section: Introductionmentioning

confidence: 99%

BP-STDP: Approximating backpropagation using spike timing dependent plasticity

2019

Self Cite

View full text Add to dashboard Cite

The problem of training spiking neural networks (SNNs) is a necessary precondition to understanding computations within the brain, a field still in its infancy. Previous work has shown that supervised learning in multi-layer SNNs enables bio-inspired networks to recognize patterns of stimuli through hierarchical feature acquisition. Although gradient descent has shown impressive performance in multi-layer (and deep) SNNs, it is generally not considered biologically plausible and is also computationally expensive. This paper proposes a novel supervised learning approach based on an event-based spike-timing-dependent plasticity (STDP) rule embedded in a network of integrate-and-fire (IF) neurons. The proposed temporally local learning rule follows the backpropagation weight change updates applied at each time step. This approach enjoys benefits of both accurate gradient descent and temporally local, efficient STDP. Thus, this method is able to address some open questions regarding accurate and efficient computations that occur in the brain. The experimental results on the XOR problem, the Iris data, and the MNIST dataset demonstrate that the proposed SNN performs as successfully as the traditional NNs. Our approach also compares favorably with the state-of-the-art multi-layer SNNs.

show abstract

A spiking network that learns to extract spike signatures from speech signals

Cited by 48 publications

References 32 publications

Deep learning in spiking neural networks

Deep learning in spiking neural networks

Predicting Performance using Approximate State Space Model for Liquid State Machines

BP-STDP: Approximating backpropagation using spike timing dependent plasticity

Contact Info

Product

Resources

About