A time-to-first-spike coding and conversion aware training for energy-efficient deep spiking neural network processor design

Lew, Dongwoo; Lee, Kyungchul; Park, Jongsun

doi:10.1145/3489517.3530457

Cited by 8 publications

(6 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most of the ANN2SNN methods are based on rate encoding and need far more time steps, e.g., T ≥ 128, than the surrogate gradient method. Conversion methods based on latency encoding rather than rate coding are also reported (93). Note that recent research has achieved a substantial reduction of T, such as surrogate learning methods (94) with T = 5 and ANN2SNN methods (51,95) with T ≤ 64.…”

Section: High-performance Simulationmentioning

confidence: 99%

SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence

Fang,

Chen,

Ding

et al. 2023

Sci. Adv.

View full text Add to dashboard Cite

Spiking neural networks (SNNs) aim to realize brain-inspired intelligence on neuromorphic chips with high energy efficiency by introducing neural dynamics and spike properties. As the emerging spiking deep learning paradigm attracts increasing interest, traditional programming frameworks cannot meet the demands of the automatic differentiation, parallel computation acceleration, and high integration of processing neuromorphic datasets and deployment. In this work, we present the SpikingJelly framework to address the aforementioned dilemma. We contribute a full-stack toolkit for preprocessing neuromorphic datasets, building deep SNNs, optimizing their parameters, and deploying SNNs on neuromorphic chips. Compared to existing methods, the training of deep SNNs can be accelerated 11×, and the superior extensibility and flexibility of SpikingJelly enable users to accelerate custom models at low costs through multilevel inheritance and semiautomatic code generation. SpikingJelly paves the way for synthesizing truly energy-efficient SNN-based machine intelligence systems, which will enrich the ecology of neuromorphic computing.

show abstract

Section: High-performance Simulationmentioning

confidence: 99%

SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence

Fang,

Chen,

Ding

et al. 2023

Sci. Adv.

View full text Add to dashboard Cite

show abstract

“…Although significant power and energy savings can be achieved by using TTFS-based SNNs, TTFS-based SNNs that are constructed by either training from scratch (Mostafa, 2016;Comsa et al, 2020) or converting from the pre-trained ANN (Rueckauer and Liu, 2018;Lew et al, 2022) tend to not perform as well as their ANN counterparts in terms of the classification accuracy. As demonstrated in a recent study (Rueckauer and Liu, 2018), converting from ANNs to TTFS-based SNNs, unfortunately, leads to accumulated approximation errors, which results in significantly lower accuracy in the SNNs than the equivalent ANNs, particularly in larger network architectures.…”

Section: Training Of Ttfs-based Snnsmentioning

confidence: 99%

“…Although there are some temporal-based works that propose solutions with higher accuracy on CIFAR and ImageNet (Park et al, 2020;Stöckl and Maass, 2021), these spiking mechanisms require much more complex hardware design with much higher energy consumption. For example, high accuracy algorithms (Stöckl and Maass, 2021;Lew et al, 2022) require updating neuron potentials dynamically, which leads to complex logic and higher memory access counts. The original TTFS-based SNN algorithm, which is the main workload of this work, is currently unable to achieve comparable accuracy on these complex datasets.…”

Section: Training Networkmentioning

confidence: 99%

A TTFS-based energy and utilization efficient neuromorphic CNN accelerator

Xiang

Srivatsa

et al. 2023

Front. Neurosci.

View full text Add to dashboard Cite

Spiking neural networks (SNNs), which are a form of neuromorphic, brain-inspired AI, have the potential to be a power-efficient alternative to artificial neural networks (ANNs). Spikes that occur in SNN systems, also known as activations, tend to be extremely sparse, and low in number. This minimizes the number of data accesses typically needed for processing. In addition, SNN systems are typically designed to use addition operations which consume much less energy than the typical multiply and accumulate operations used in DNN systems. The vast majority of neuromorphic hardware designs support rate-based SNNs, where the information is encoded by spike rates. Generally, rate-based SNNs can be inefficient as a large number of spikes will be transmitted and processed during inference. One coding scheme that has the potential to improve efficiency is the time-to-first-spike (TTFS) coding, where the information isn't presented through the frequency of spikes, but instead through the relative spike arrival time. In TTFS-based SNNs, each neuron can only spike once during the entire inference process, and this results in high sparsity. The activation sparsity of TTFS-based SNNs is higher than rate-based SNNs, but TTFS-based SNNs have yet to achieve the same accuracy as rate-based SNNs. In this work, we propose two key improvements for TTFS-based SNN systems: (1) a novel optimization algorithm to improve the accuracy of TTFS-based SNNs and (2) a novel hardware accelerator for TTFS-based SNNs that uses a scalable and low-power design. Our work in TTFS coding and training improves the accuracy of TTFS-based SNNs to achieve state-of-the-art results on the MNIST and Fashion-MNIST datasets. Meanwhile, our work reduces the power consumption by at least 2.4×, 25.9×, and 38.4× over the state-of-the-art neuromorphic hardware on MNIST, Fashion-MNIST, and CIFAR10, respectively.

show abstract

“…As benchmarks, pre-trained VGG-16, ResNet-18 and ResNet-34 models on CIFAR-10, CIFAR-100 and ImageNet are directly converted to one-spike SNNs. The major difference between the one-spike SNN and the prior work [24], [25], [32] is that the conversion process does not require any constraints on ANNs or conversion-aware ANN training. In [16], [22], conversion was performed by removing batch normalization layers or bias terms resulting in lower accuracy.…”

Section: A Experimental Setupmentioning

confidence: 99%

Comparison and Selection of Spike Encoding Algorithms for SNN on FPGA

Wang

Hao

Wang

et al. 2023

IEEE Trans. Biomed. Circuits Syst.

View full text Add to dashboard Cite

As spiking neural networks (SNNs) are event-driven, energy efficiency is higher than conventional artificial neural networks (ANNs). Since SNN delivers data through discrete spikes, it is difficult to use gradient methods for training, limiting its accuracy. To keep the accuracy of SNNs similar to ANN counterparts, pre-trained ANNs are converted to SNNs (ANNto-SNN conversion). During the conversion, encoding activations of ANNs to a set of spikes in SNNs is crucial for minimizing the conversion loss. In this work, we propose a single-spike phase coding as an encoding scheme that minimizes the number of spikes to transfer data between SNN layers. To minimize the encoding error due to single-spike approximation in phase coding, threshold shift and base manipulation are proposed. Without any additional retraining or architectural constraints on ANNs, the proposed conversion method does not lose inference accuracy (0.58% on average) verified on three convolutional neural networks (CNNs) with CIFAR and ImageNet datasets. In addition, graph convolutional networks (GCNs) are converted to SNNs successfully with an average accuracy loss of 0.90%. Most importantly, the energy efficiency of our SNN improves by 4.6∼17.3× compared to the ANN baseline.

show abstract

A time-to-first-spike coding and conversion aware training for energy-efficient deep spiking neural network processor design

Cited by 8 publications

References 15 publications

SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence

SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence

A TTFS-based energy and utilization efficient neuromorphic CNN accelerator

Comparison and Selection of Spike Encoding Algorithms for SNN on FPGA

Contact Info

Product

Resources

About