Memory via Temporal Delays in weightless Spiking Neural Network

Hazan, Hananel; Caby, Simon; Earl, Christopher; Siegelmann, Hava T.; Levin, Michael

doi:10.48550/arxiv.2202.07132

Cited by 3 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This was also presented by [205], who proposed a supervised delay learning algorithm for spiking neurons with temporal encoding, in which both the weight and delay of a synaptic connection can be adjusted to enhance the learning performance. Other models, such as that of [206], propose a weightless spiking neural networks that can perform a simple classification task which is applied to MNIST. In a recent paper [207], the authors proposed a gradient descent-based learning algorithm for synaptic delays to enhance the sequential learning performance of a single spiking neuron.…”

Section: Learning Synaptic Delaysmentioning

confidence: 99%

Precise Spiking Motifs in Neurobiological and Neuromorphic Data

Grimaldi

Gruel²,

Besnainou

et al. 2022

Brain Sciences

View full text Add to dashboard Cite

Why do neurons communicate through spikes? By definition, spikes are all-or-none neural events which occur at continuous times. In other words, spikes are on one side binary, existing or not without further details, and on the other, can occur at any asynchronous time, without the need for a centralized clock. This stands in stark contrast to the analog representation of values and the discretized timing classically used in digital processing and at the base of modern-day neural networks. As neural systems almost systematically use this so-called event-based representation in the living world, a better understanding of this phenomenon remains a fundamental challenge in neurobiology in order to better interpret the profusion of recorded data. With the growing need for intelligent embedded systems, it also emerges as a new computing paradigm to enable the efficient operation of a new class of sensors and event-based computers, called neuromorphic, which could enable significant gains in computation time and energy consumption—a major societal issue in the era of the digital economy and global warming. In this review paper, we provide evidence from biology, theory and engineering that the precise timing of spikes plays a crucial role in our understanding of the efficiency of neural networks.

show abstract

Section: Learning Synaptic Delaysmentioning

confidence: 99%

Precise Spiking Motifs in Neurobiological and Neuromorphic Data

Grimaldi

Gruel²,

Besnainou

et al. 2022

Brain Sciences

View full text Add to dashboard Cite

show abstract

“…This was also presented by [186] which propose a supervised delay learning algorithm for spiking neurons with temporal encoding, in which both the weight and delay of a synaptic connection can be adjusted to enhance the learning performance. Other models like that of [187] propose a weightless spiking neural networks that can perform a simple classification task which is applied to MNIST. In a recent paper [188], authors propose a gradient descent-based learning algorithm for synaptic delays to enhance the sequential learning performance of a single spiking neuron.…”

Section: Learning Synaptic Delaysmentioning

confidence: 99%

Precise Spiking Motifs in Neurobiological and Neuromorphic Data

Grimaldi¹,

Gruel²,

Besnainou³

et al. 2022

Preprint

View full text Add to dashboard Cite

Why do neurons communicate through spikes? By definition, spikes are all-or-none neural events which occur at continuous times. In other words, spikes are on one side binary, existing or not without further details, and on the other can occur at any asynchronous time, without the need for a centralized clock. This stands in stark contrast to the analog representation of values and the discretized timing classically used in digital processing and at the base of modern-day neural networks. As neural systems almost systematically use this so-called event-based representation in the living world, a better understanding of this phenomenon remains a fundamental challenge in neurobiology in order to better interpret the profusion of recorded data. With the growing need for intelligent embedded systems, it also emerges as a new computing paradigm to enable the efficient operation of a new class of sensors and event-based computers, called neuromorphic, which could enable significant gains in computation time and energy consumption — a major societal issue in the era of the digital economy and global warming. In this review paper, we provide evidence from biology, theory and engineering that the precise timing of spikes plays a crucial role in our understanding of the efficiency of neural networks.

show abstract

Efficient sparse spiking auto-encoder for reconstruction, denoising and classification

Walters,

Rahimian Kalatehbali,

Cai

et al. 2024

Neuromorph. Comput. Eng.

View full text Add to dashboard Cite

Auto-encoders are capable of performing input reconstruction, denoising, and classification through an encoder-decoder structure. Spiking Auto- Encoders (SAEs) can utilize asynchronous sparse spikes to improve power efficiency and processing latency on neuromorphic hardware. In this work, we propose an efficient SAE trained using only Spike-Timing-Dependant Plasticity (STDP) learning. Our auto-encoder uses the Time-To-First-Spike (TTFS) encoding scheme and needs to update all synaptic weights only once per input, promoting both training and inference efficiency due to the extreme sparsity. We showcase robust reconstruction performance on the Modified National Institute of Standards and Technology (MNIST) and Fashion-MNIST datasets with significantly fewer spikes compared to state-of-the-art SAEs by 1 − 3 orders of magnitude. Moreover, we achieve robust noise reduction results on the MNIST dataset. When the same noisy inputs are used for classification, accuracy degradation is reduced by 30 − 80% compared to prior works. It also exhibits classification accuracies comparable to previous STDP-based classifiers, while remaining competitive with other backpropagation-based spiking classifiers that require global learning through gradients and significantly more spikes for encoding and classification of MNIST/Fashion-MNIST inputs. The presented results demonstrate a promising pathway towards building efficient sparse spiking auto-encoders with local learning, making them highly suited for hardware integration.

show abstract

Memory via Temporal Delays in weightless Spiking Neural Network

Cited by 3 publications

References 0 publications

Precise Spiking Motifs in Neurobiological and Neuromorphic Data

Precise Spiking Motifs in Neurobiological and Neuromorphic Data

Precise Spiking Motifs in Neurobiological and Neuromorphic Data

Efficient sparse spiking auto-encoder for reconstruction, denoising and classification

Contact Info

Product

Resources

About