GLSNN: A Multi-Layer Spiking Neural Network Based on Global Feedback Alignment and Local STDP Plasticity

Zhao, Dongcheng; Zeng, Yi; Zhang, Tielin; Shi, Mengting; Zhao, Feifei

doi:10.3389/fncom.2020.576841

Cited by 50 publications

(25 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This plasticity is biologically-plausible and will also be used as the main credit assignment of SNNs in our NRR-SNN algorithm. Zhao et al (2020) have proposed a similar method, where global random feedback alignment is combined with STDP for efficient credit assignment.…”

Section: Related Workmentioning

confidence: 99%

Neuronal-Plasticity and Reward-Propagation Improved Recurrent Spiking Neural Networks

et al. 2021

Self Cite

View full text Add to dashboard Cite

Different types of dynamics and plasticity principles found through natural neural networks have been well-applied on Spiking neural networks (SNNs) because of their biologically-plausible efficient and robust computations compared to their counterpart deep neural networks (DNNs). Here, we further propose a special Neuronal-plasticity and Reward-propagation improved Recurrent SNN (NRR-SNN). The historically-related adaptive threshold with two channels is highlighted as important neuronal plasticity for increasing the neuronal dynamics, and then global labels instead of errors are used as a reward for the paralleling gradient propagation. Besides, a recurrent loop with proper sparseness is designed for robust computation. Higher accuracy and stronger robust computation are achieved on two sequential datasets (i.e., TIDigits and TIMIT datasets), which to some extent, shows the power of the proposed NRR-SNN with biologically-plausible improvements.

show abstract

Section: Related Workmentioning

confidence: 99%

Neuronal-Plasticity and Reward-Propagation Improved Recurrent Spiking Neural Networks

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…NALSM8000 achieved a top accuracy of 97.61% (97.49 ± 0.11%) on MNIST, 97.51% (97.42 ± 0.07%) on N-MNIST, and 85.84% (85.61 ± 0.18%) on Fashion-MNIST. Compared to previously reported benchmarks on MNIST and Fashion-MNIST, the NALSM8000 outperformed all brain-inspired learning methods that do not use backpropagation of gradients or its approximation through feedback alignment [72,73], with the exception of [74] for MNIST. While [74] demonstrated that a fully-connected 2-layer spiking network can achieve high accuracy through a combination of biologically-plausible plasticity rules, it is not clear how such an approach would scale to more layers without some form of backpropagation.…”

Section: Larger Liquids Increased Nalsm Accuracymentioning

confidence: 71%

Increasing Liquid State Machine Performance with Edge-of-Chaos Dynamics Organized by Astrocyte-modulated Plasticity

Иванов¹,

Michmizos²

2021

Preprint

View full text Add to dashboard Cite

The liquid state machine (LSM) combines low training complexity and biological plausibility, which has made it an attractive machine learning framework for edge and neuromorphic computing paradigms. Originally proposed as a model of brain computation, the LSM tunes its internal weights without backpropagation of gradients, which results in lower performance compared to multi-layer neural networks. Recent findings in neuroscience suggest that astrocytes, a long-neglected non-neuronal brain cell, modulate synaptic plasticity and brain dynamics, tuning brain networks to the vicinity of the computationally optimal critical phase transition between order and chaos. Inspired by this disruptive understanding of how brain networks self-tune, we propose the neuron-astrocyte liquid state machine (NALSM) 1 that addresses under-performance through self-organized near-critical dynamics. Similar to its biological counterpart, the astrocyte model integrates neuronal activity and provides global feedback to spike-timing-dependent plasticity (STDP), which self-organizes NALSM dynamics around a critical branching factor that is associated with the edge-of-chaos. We demonstrate that NALSM achieves state-of-the-art accuracy versus comparable LSM methods, without the need for data-specific hand-tuning. With a top accuracy of 97.61% on MNIST, 97.51% on N-MNIST, and 85.84% on Fashion-MNIST, NALSM achieved comparable performance to current fully-connected multi-layer spiking neural networks trained via backpropagation. Our findings suggest that the further development of braininspired machine learning methods has the potential to reach the performance of deep learning, with the added benefits of supporting robust and energy-efficient neuromorphic computing on the edge.

show abstract

“…Interestingly, incorporating the neuromorphic principles comes at the cost of a decrease in performance for the existing methods. For example, methods that employ continuous feedback computations (Zhao et al 2020;Lillicrap et al 2016) outperformed those that have event-based feedback (Guerguiev, Lillicrap, and Richards 2017;Tavanaei and Maida 2019;Shrestha et al 2019). However, our method achieved a high level of performance without sacrificing any of the neuromorphic principles.…”

Section: Comparing Biograd Against Other Biologically Plausible Appro...mentioning

confidence: 89%

“…The need for separate feedback pathways is alleviated in (Zhao et al 2020) by not only propagating errors directly from the output layer to hidden layers (Nøkland 2016), but also employing multi-compartment neurons that receive feedforward and feedback information in segregated compartments. However, these methods propagate continuousvalued errors, violating the event-based nature of computations.…”

Section: Related Workmentioning

confidence: 99%

BioGrad: Biologically Plausible Gradient-Based Learning for Spiking Neural Networks

Tang¹,

Kumar²,

Polykretis³

et al. 2021

Preprint

View full text Add to dashboard Cite

Spiking neural networks (SNN) have started to deliver energy-efficient, massively parallel, and low-latency solutions to AI problems, facilitated by the emerging neuromorphic hardware. To harness these computational benefits, SNN need to be trained by learning algorithms that adhere to braininspired neuromorphic principles, namely event-based, local, and online computations. However, the state-of-the-art SNN training algorithms are based on backpropagation that does not follow the above neuromorphic computational principles. Due to its limited biological plausibility, the application of backprop to SNN requires non-local feedback pathways for transmitting continuous-valued errors, and relies on gradients from future timesteps. The recent introduction of biologically plausible modifications to backprop has helped overcome several of its limitations, but limits the degree to which backprop is approximated, which hinders its performance. Here, we propose a biologically plausible gradientbased learning algorithm for SNN that is functionally equivalent to backprop, while adhering to all three neuromorphic computational principles. We introduced multi-compartment spiking neurons with local eligibility traces to compute the gradients required for learning, and a periodic "sleep" phase to further improve the approximation to backprop during which a local Hebbian rule aligns the feedback and feedforward weights. Our method achieved the same level of performance as backprop with multi-layer fully connected SNN on MNIST (98.13%) and the event-based N-MNIST (97.59%) datasets. We then deployed our learning algorithm on Intel's Loihi neuromorphic processor to train a 1-hidden-layer network for MNIST, and obtained 93.32% test accuracy while consuming 400 times less energy per training sample than Bi-oGrad on GPU. Our work demonstrates that optimal learning is feasible in neuromorphic computing, and further pursuing its biological plausibility can better capture the computational benefits of this emerging computing paradigm.

show abstract

GLSNN: A Multi-Layer Spiking Neural Network Based on Global Feedback Alignment and Local STDP Plasticity

Cited by 50 publications

References 43 publications

Neuronal-Plasticity and Reward-Propagation Improved Recurrent Spiking Neural Networks

Neuronal-Plasticity and Reward-Propagation Improved Recurrent Spiking Neural Networks

Increasing Liquid State Machine Performance with Edge-of-Chaos Dynamics Organized by Astrocyte-modulated Plasticity

BioGrad: Biologically Plausible Gradient-Based Learning for Spiking Neural Networks

Contact Info

Product

Resources

About