Recognizing Images with at most one Spike per Neuron

Stöckl, Christoph; Maass, Wolfgang

doi:10.48550/arxiv.2001.01682

Cited by 3 publications

(6 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Alternatively, other approaches such as [8,16] can be applied to any type of network. The first one manages to do this by using circuits of neurons in order to approximate arbitrary functions.…”

Section: Conversion Methodsmentioning

confidence: 99%

“…In order to overcome the aforementioned challenges, some approaches use conversion methods [6][7][8], where they train non-spiking ANNs and then approximate their computations using an SNN. Compared to directly training an SNN, these methods are not able to perform online learning, they lose temporal resolution, and in most cases they have higher latency and energy consumption.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Keys to accurate feature extraction using residual spiking neural networks

Vicente-Sola

Manna

Kirkland

et al. 2022

Neuromorph. Comput. Eng.

View full text Add to dashboard Cite

Spiking neural networks (SNNs) have become an interesting alternative to conventional artificial neural networks (ANN) thanks to their temporal processing capabilities and energy efficient implementations in neuromorphic hardware. However the challenges involved in training SNNs have limited their performance in terms of accuracy and thus their applications. Improving learning algorithms and neural architectures for a more accurate feature extraction is therefore one of the current priorities in SNN research. In this paper we present a study on the key components of modern spiking architectures. We design a spiking version of the successful residual network architecture and provide an in-depth study on the possible implementations of spiking residual connections. This study shows how, depending on the use case, the optimal residual connection implementation may vary. Additionally, we empirically compare different techniques in image classification datasets taken from the best performing networks. Our results provide a state of the art guide to SNN design, which allows to make informed choices when trying to build the optimal visual feature extractor. Finally, our network outperforms previous SNN architectures in CIFAR-10 (94.14%) and CIFAR-100 (74.65%) datasets and matches the state of the art in DVS-CIFAR10 (72.98%), with less parameters than the previous state of the art and without the need for ANN-SNN conversion.

show abstract

Section: Conversion Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Keys to accurate feature extraction using residual spiking neural networks

Vicente-Sola

Manna

Kirkland

et al. 2022

Neuromorph. Comput. Eng.

View full text Add to dashboard Cite

show abstract

“…Alternatively, other approaches such as [8,12] can be applied to any type of network. The first one manages to do this by using circuits of neurons in order to approximate arbitrary functions.…”

Section: Conversion Methodsmentioning

confidence: 99%

“…In order to overcome the aforementioned challenges, some approaches use conversion methods [6,7,8], where they train non-spiking ANNs and then approximate their computations using an SNN. Compared to directly training an SNN, these methods have higher latency and energy consumption, they are not able to perform online learning and lose temporal resolution.…”

Section: Introductionmentioning

confidence: 99%

Keys to Accurate Feature Extraction Using Residual Spiking Neural Networks

Vicente-Sola,

Manna,

Kirkland

et al. 2021

Preprint

View full text Add to dashboard Cite

Spiking neural networks (SNNs) have become an interesting alternative to conventional artificial neural networks (ANN) thanks to their temporal processing capabilities and their low-SWaP (Size, Weight, and Power) and energy efficient implementations in neuromorphic hardware. However the challenges involved in training SNNs have limited their performance in terms of accuracy and thus their applications. Improving learning algorithms and neural architectures for a more accurate feature extraction is therefore one of the current priorities in SNN research. In this paper we present a study on the key components of modern spiking architectures. We empirically compare different techniques in image classification datasets taken from the best performing networks. We design a spiking version of the successful residual network (ResNet) architecture and test different components and training strategies on it. Our results provide a state of the art guide to SNN design, which allows to make informed choices when trying to build the optimal visual feature extractor. Finally, our network outperforms previous SNN architectures in and CIFAR-100 (74.5%) datasets and matches the state of the art in DVS-CIFAR10 (71.3%), with less parameters than the previous state of the art and without the need for ANN-SNN conversion. Code available at https://github. com/VicenteAlex/Spiking_ResNet

show abstract

“…Conversion approaches focus mostly on computer vision domain and CNN-to-SNN conversion [ 43 ]; however, there has been an attempt at converting recurrent neural networks as well [ 59 ]. Resulting SNNs usually are rate-coded, but there have been attempts to use latency coding as well [ 60 ]. Most conversion approaches are based on converting neurons with rectified linear unit (ReLU) activation function to IF neurons.…”

Section: Types Of Learning Approachesmentioning

confidence: 99%

Overview of Spiking Neural Network Learning Approaches and Their Computational Complexities

Pietrzak

Szczęsny

Huderek

et al. 2023

Sensors

View full text Add to dashboard Cite

Spiking neural networks (SNNs) are subjects of a topic that is gaining more and more interest nowadays. They more closely resemble actual neural networks in the brain than their second-generation counterparts, artificial neural networks (ANNs). SNNs have the potential to be more energy efficient than ANNs on event-driven neuromorphic hardware. This can yield drastic maintenance cost reduction for neural network models, as the energy consumption would be much lower in comparison to regular deep learning models hosted in the cloud today. However, such hardware is still not yet widely available. On standard computer architectures consisting mainly of central processing units (CPUs) and graphics processing units (GPUs) ANNs, due to simpler models of neurons and simpler models of connections between neurons, have the upper hand in terms of execution speed. In general, they also win in terms of learning algorithms, as SNNs do not reach the same levels of performance as their second-generation counterparts in typical machine learning benchmark tasks, such as classification. In this paper, we review existing learning algorithms for spiking neural networks, divide them into categories by type, and assess their computational complexity.

show abstract

Recognizing Images with at most one Spike per Neuron

Cited by 3 publications

References 15 publications

Keys to accurate feature extraction using residual spiking neural networks

Keys to accurate feature extraction using residual spiking neural networks

Keys to Accurate Feature Extraction Using Residual Spiking Neural Networks

Overview of Spiking Neural Network Learning Approaches and Their Computational Complexities

Contact Info

Product

Resources

About