Fast and Efficient Information Transmission with Burst Spikes in Deep Spiking Neural Networks

Abstract

The spiking neural networks (SNNs) are considered as one of the most promising artificial neural networks due to their energyefficient computing capability. Recently, conversion of a trained deep neural network to an SNN has improved the accuracy of deep SNNs. However, most of the previous studies have not achieved satisfactory results in terms of inference speed and energy efficiency. In this paper, we propose a fast and energy-efficient information transmission method with burst spikes and hybrid neural coding scheme in deep SNNs. Our experimental results showed the proposed methods can improve inference energy efficiency and shorten the latency.

Boosting Throughput and Efficiency of Hardware Spiking Neural Accelerators Using Time Compression Supporting Multiple Spike Codes

“…The LIF model for burst coding is also based on (3) (Park et al, 2019). A bursting function g i (t) is introduced to implement the bursting behavior per each presynaptic neuron i (Park et al, 2019):…”

“…We redesign our TC-SNN accelerators using bursting IOW LIF models to support burst coding (Park et al, 2019) and compare their performances with the baseline on the TI46 speech dataset and CityScape image dataset in Table 3. Once again, the proposed time compression leads to large runtime and energy reductions and the degradation of classification accuracy is graceful.…”

“…We demonstrate the proposed TC-SNN and PTC-SNN compression architectures by realizing several liquid-state machine (LSM) spiking neural accelerators with a time compression ratio up to 16:1 on a Xilinx Zynq-7000 FPGA. Using the TI46 Speech Corpus (Liberman et al, 1991), the CityScape image recognition dataset (Cordts et al, 2016), and N-TIDIGITS18 dataset (Anumula et al, 2018), we demonstrate the feasibility of supporting large time compression ratios of up to 16×, delivering up to 15.93×, 13.88×, and 86.21× improvements in throughput, energy dissipation, the tradeoffs between hardware area, runtime, energy, and classification accuracy, respectively based on various spike coding mechanisms including burst coding (Park et al, 2019) on a Xilinx Zynq-7000 FPGA. These results are achieved while incurring little extra hardware overhead.…”

“…SNNs support various rate/temporal spike codes among which rate coding using Poisson spike trains is popular. However, in that case, the low-power advantage of SNNs may be offset by long latency during which many spikes are processed for ensuring decision accuracy (Kim et al, 2018;Park et al, 2019). Various temporal codes have been attempted to improve the efficiency of information representation (Thorpe, 1990;Thorpe et al, 2001;Izhikevich, 2002;Kayser et al, 2009;Kim et al, 2018).…”

Self-Evolutionary Neuron Model for Fast-Response Spiking Neural Networks

“…[18] proposed a weight normalization method named "modelbased normalization" and "data-based normalization" to help regulate the firing rates of spiking feed-forward networks and spiking convolutional networks, and their pattern recognition experiments showed that the normalization technique boosts the convergence speed of the firing activity of spiking neurons, which improves the real-time performance. In [19] the authors proposed an information transmission method with burst spikes and a layer-wise hybrid neural coding scheme for deep SNNs, and their experiment results of image classification tasks proved that the proposed method substantially improves the inference efficiency in terms of speed and energy, while also maintains reasonable accuracy. In [20], Zambrano et al presented an adaptive spiking neurons based network, where the neurons encode information in spike-trains using a form of Asynchronous Pulsed Sigma-Delta coding, and the authors demonstrated that the proposed neuron models based network responds an order of magnitude faster and uses an order of magnitude fewer spikes.…”