Spiking neural network reinforcement learning method based on temporal coding and STDP

Sboev, Alexander; Vlasov, Danila; Rybka, Roman; Serenko, Alexey

doi:10.1016/j.procs.2018.11.107

Cited by 7 publications

(4 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…accuracy SNNs are not inferior to ANNs but often even better. In [26] SNNs have shown comparable results to ANN and ML for the classification of the Iris data set. Same has been shown for the MNIST data set in comparison with a CNN [16], whereas the letters have been recorded with a retinomorphic data set [22] for the SNN.…”

Section: Advantages and Drawbacksmentioning

confidence: 88%

Learning algorithms for spiking neural networks: should one use learning algorithms from ANN/DL or neurological plausible learning? - A thought-provoking impulse

Bogdan¹

2022

XLIII Jornadas De Automática: Libro De Actas: 7, 8 Y 9 De Septiembre De 2022, Logroño (La Rioja)

View full text Add to dashboard Cite

Artificial Neural Networks (ANN) and Machine Learning (ML) currently also known as Deep Learning (DL) became more and more important in industrial applications during the last decade. This is due to new possibilities by strongly increased available computational power in connection with a renaissance of ANN in terms of so-called Deep Learning (DL). As DL requires especially for Big Data extreme computational power, the question of resource preserving methods came recently into the focus. Also, the often propagated intelligence of DL resp. "Cognitive Computing" in terms of contextual information processing is more often discussed since it is effectively missed in DL solutions. One option to overcome both challenges might be the third generation of ANNs: Spiking Neural Networks (SNN). But since SNN training methods are slow compared to DL learning algorithms, the question of the way how to learn SNNs arose. We will discuss different aspects of learning algorithms for SNNs: Is it useful to adopt DL learning algorithms to SNN or not, especially if one will preserve the "cognitive" functions of SNNs?

show abstract

Section: Advantages and Drawbacksmentioning

confidence: 88%

Learning algorithms for spiking neural networks: should one use learning algorithms from ANN/DL or neurological plausible learning? - A thought-provoking impulse

Bogdan¹

2022

XLIII Jornadas De Automática: Libro De Actas: 7, 8 Y 9 De Septiembre De 2022, Logroño (La Rioja)

View full text Add to dashboard Cite

show abstract

“…Rate coding uses the firing rate of spike trains in a time window to encode information, where input real numbers are converted into spike trains with a frequency proportional to the input value (Cheng et al 2020a). Temporal coding encodes information with the relative timing of individual spikes, where input values are usually converted into spike trains with the precise time (Comsa et al 2020;Sboev et al 2018). Besides that, population coding is special in integrating these two types.…”

Section: Information Coding Methods In Snnsmentioning

confidence: 99%

Multi-Sacle Dynamic Coding Improved Spiking Actor Network for Reinforcement Learning

Zhang

Jia

et al. 2022

AAAI

View full text Add to dashboard Cite

With the help of deep neural networks (DNNs), deep reinforcement learning (DRL) has achieved great success on many complex tasks, from games to robotic control. Compared to DNNs with partial brain-inspired structures and functions, spiking neural networks (SNNs) consider more biological features, including spiking neurons with complex dynamics and learning paradigms with biologically plausible plasticity principles. Inspired by the efficient computation of cell assembly in the biological brain, whereby memory-based coding is much more complex than readout, we propose a multiscale dynamic coding improved spiking actor network (MDC-SAN) for reinforcement learning to achieve effective decision-making. The population coding at the network scale is integrated with the dynamic neurons coding (containing 2nd-order neuronal dynamics) at the neuron scale towards a powerful spatial-temporal state representation. Extensive experimental results show that our MDC-SAN performs better than its counterpart deep actor network (based on DNNs) on four continuous control tasks from OpenAI gym. We think this is a significant attempt to improve SNNs from the perspective of efficient coding towards effective decision-making, just like that in biological networks.

show abstract

“…Unlike LIF neurons, the DNs showed a higher complexity with an additional implicit U , making the dynamical changing of equilibrium points different. The little differences of U would cause a big update of V according to the definition of DNs, especially when the parameter θ b was small in Equation (19). Hence, the DNs would not only show similar firing patterns with the positive strong stimulus but also exhibit a sparse firing with the weak-positive and negative stimulus, instead of stopping firing like LIF neurons.…”

Section: Learn and Analyze Dnsmentioning

confidence: 99%

“…There are two main categories of information encoding (rate and temporal types) at the input coding scale of SNN. Rate coding uses the firing rate of spike trains in a time window to encode information, where input real numbers are converted into spike trains with a frequency proportional to the input value [14,15], and temporal coding encodes information with the relative timing of individual spikes, where input values are usually converted into spike trains with the precise time [16,17,18,19]. Besides that, population coding is special in integrating these two types.…”

Section: Introductionmentioning

confidence: 99%

Population-coding and Dynamic-neurons improved Spiking Actor Network for Reinforcement Learning

Zhang¹,

Zhang²,

Jia³

et al. 2021

Preprint

View full text Add to dashboard Cite

With the Deep Neural Networks (DNNs) as a powerful function approximator, Deep Reinforcement Learning (DRL) has been excellently demonstrated on robotic control tasks. Compared to DNNs with vanilla artificial neurons, the biologically plausible Spiking Neural Network (SNN) contains a diverse population of spiking neurons, making it naturally powerful on state representation with spatial and temporal information. Based on a hybrid learning framework, where a spike actor-network infers actions from states and a deep critic network evaluates the actor, we propose a Population-coding and Dynamic-neurons improved Spiking Actor Network (PDSAN) for efficient state representation from two different scales: input coding and neuronal coding. For input coding, we apply population coding with dynamically receptive fields to directly encode each input state component. For neuronal coding, we propose different types of dynamic-neurons (containing 1st-order and 2nd-order neuronal dynamics) to describe much more complex neuronal dynamics. Finally, the PDSAN is trained in conjunction with deep critic networks using the Twin Delayed Deep Deterministic policy gradient algorithm (TD3-PDSAN). Extensive experimental results show that our TD3-PDSAN model achieves better performance than state-of-the-art models on four OpenAI gym benchmark tasks. It is an im- * Duzhen Zhang and Tielin Zhang are co-first authors of this paper * *

show abstract

Spiking neural network reinforcement learning method based on temporal coding and STDP

Cited by 7 publications

References 11 publications

Learning algorithms for spiking neural networks: should one use learning algorithms from ANN/DL or neurological plausible learning? - A thought-provoking impulse

Learning algorithms for spiking neural networks: should one use learning algorithms from ANN/DL or neurological plausible learning? - A thought-provoking impulse

Multi-Sacle Dynamic Coding Improved Spiking Actor Network for Reinforcement Learning

Population-coding and Dynamic-neurons improved Spiking Actor Network for Reinforcement Learning

Contact Info

Product

Resources

About