Asynchronous Spiking Neurons, the Natural Key to Exploit Temporal Sparsity

Abstract

Inference of Deep Neural Networks for stream signal (Video/Audio) processing in edge devices is still challenging. Unlike the most state of the art inference engines which are efficient for static signals, our brain is optimized for real-time dynamic signal processing. We believe one important feature of the brain (asynchronous state-full processing) is the key to its excellence in this domain. In this work, we show how asynchronous processing with state-full neurons allows exploitation of the existing sparsity in natural signals. This paper explains three different types of sparsity and proposes an inference algorithm which exploits all types of sparsities in the execution of already trained networks. Our experiments in three different applications (Handwritten digit recognition, Autonomous Steering and Hand-Gesture recognition) show that this model of inference reduces the number of required operations for sparse input data by a factor of one to two orders of magnitudes. Additionally, due to fully asynchronous processing this type of inference can be run on fully distributed and scalable neuromorphic hardware platforms.

Neural Network Optimization for Reinforcement Learning Tasks Using Sparse Computations

“…The main idea behind this approachn [14,4] ] is to use temporal sparsity when working with sequential data.…”

“…Authors of [4,14] call this approach "Hysteresis Quantizer". To implement it, it is necessary to introduce an additional variable into each neuron; such variable will be used to record last transmitted value.…”

MOSDA: On-Chip Memory Optimized Sparse Deep Neural Network Accelerator With Efficient Index Matching

“…This paper mainly focuses on the sparsity within one frame, which is also called spatial sparsity. Besides the spatial sparsity, there is also temporal sparsity among frames [43]. If data rarely change over time, data is regarded as temporarily sparse.…”

How to exploit sparsity in RNNs on event-driven architectures

“…The addition of thresholding logic to suppress propagation of small deltas reduces computation counts even further [11]. Sparse execution of neural networks is far more efficient on asynchronous architectures [22], than on statically scheduled architectures such as CPUs and GPUs.…”