An Efficient Spiking Neural Network for Recognizing Gestures with a DVS Camera on the Loihi Neuromorphic Processor

Massa, Riccardo; Marchisio, Alberto; Martina, Maurizio; Shafique, Muhammad

doi:10.1109/ijcnn48605.2020.9207109

Cited by 85 publications

(33 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…First, higher layer counts demand an increasing number of time steps in order to achieve maximum accuracy. This is a well-understood and fundamental property of rate-coded feedforward networks [21], [31], [32]. Second, the need to distribute larger networks across multiple Loihi chips leads to congestion in the links between the chips and often a dramatic increase in execution time per time step.…”

Section: A Deep Snn Conversionmentioning

confidence: 99%

Advancing Neuromorphic Computing With Loihi: A Survey of Results and Outlook

et al. 2021

View full text Add to dashboard Cite

Deep artificial neural networks apply principles of the brain's information processing that led to breakthroughs in machine learning spanning many problem domains. Neuromorphic computing aims to take this a step further to chips more directly inspired by the form and function of biological neural circuits, so they can process new knowledge, adapt, behave, and learn in real time at low power levels. Despite several decades of research, until recently, very few published results have shown that today's neuromorphic chips can demonstrate quantitative computational value. This is now changing with the advent of Intel's Loihi, a neuromorphic research processor designed to support a broad range of spiking neural networks with sufficient scale, performance, and features to deliver competitive results compared to state-of-the-art contemporary computing architectures. This survey reviews results that are obtained to date with Loihi across the major algorithmic domains under study, including deep learning approaches and novel approaches that aim to more directly harness the key features of spike-based neuromorphic hardware. While conventional feedforward deep neural networks show modest if any benefit on Loihi, more brain-inspired networks using recurrence, precise spike-timing relationships, synaptic plasticity, stochasticity, and sparsity perform certain computation with orders of magnitude lower latency and energy compared to state-of-the-art conventional approaches. These compelling neuromorphic networks solve a diverse range of problems representative of brain-like computation, such as event-based

show abstract

Section: A Deep Snn Conversionmentioning

confidence: 99%

Advancing Neuromorphic Computing With Loihi: A Survey of Results and Outlook

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Due to their bio-inspired operations, SNNs have a high potential to provide energy-efficient computation. Recent works have been actively exploring two research directions, i.e., SNNs with a localized learning rule like the spike-timing-dependent plasticity (STDP) [3], and SNNs obtained from DNN conversions [89].…”

Section: Neuromorphic Research Considering Snnsmentioning

confidence: 99%

“…The Q-SpiNN [96] explores different precision levels, rounding schemes, and quantization schemes (i.e., post-and in-training quantization) to maximize memory savings for both weights and neuron parameters (which occupy considerable amount of memory in the accelerator fabric). The other techniques target at mapping and running the SNN applications (e.g., DVS Gesture Recognition [89] and Autonomous Cars [97]) on neuromorphic hardware (i.e., Loihi) to improve the energy efficiency of their processing compared to running them on conventional platforms (e.g., CPUs, GPUs). As shown in Fig.…”

Section: A Improving the Energy Efficiency Of Snnsmentioning

confidence: 99%

Towards Energy-Efficient and Secure Edge AI: A Cross-Layer Framework

Shafique,

Marchisio,

Putra

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

The security and privacy concerns along with the amount of data that is required to be processed on regular basis has pushed processing to the edge of the computing systems. Deploying advanced Neural Networks (NN), such as deep neural networks (DNNs) and spiking neural networks (SNNs), that offer state-of-the-art results on resourceconstrained edge devices is challenging due to the stringent memory and power/energy constraints. Moreover, these systems are required to maintain correct functionality under diverse security and reliability threats. This paper first discusses existing approaches to address energy efficiency, reliability, and security issues at different system layers, i.e., hardware (HW) and software (SW). Afterward, we discuss how to further improve the performance (latency) and the energy efficiency of Edge AI systems through HW/SW-level optimizations, such as pruning, quantization, and approximation. To address reliability threats (like permanent and transient faults), we highlight cost-effective mitigation techniques, like fault-aware training and mapping. Moreover, we briefly discuss effective detection and protection techniques to address security threats (like model and data corruption). Towards the end, we discuss how these techniques can be combined in an integrated cross-layer framework for realizing robust and energy-efficient Edge AI systems.

show abstract

“…Hand gesture recognition has been an active field of research due to the quest to provide a better, more efficient, and intuitive mechanism for human-computer interaction (HCI). Many different sensor modalities have been used to address this challenge, such as radar [1][2][3], cameras [4,5], dynamic vision sensors (DVS) [6][7][8][9], or electromyography (EMG) systems [10,11]. Moreover, several different machine learning techniques have been proposed to address HGR, such as a convolutional neural network (CNN) [12], long short-term memory (LSTM) [12,13], and spiking neural networks [8,11].…”

Section: Introductionmentioning

confidence: 99%

“…While advancement in image recognition systems makes frame-based cameras propitious for HGR, it has disadvantages, such as the need for proper illumination conditions, camera position, and additional image processing tasks such as segmentation to isolate the gestures from the background scene. Alternatively, event-based cameras, i.e., DVS, are designed as a neuromorphic event-based vision system and a natural fit to be used with SNNs [6][7][8][9]. However, they remain a relatively costly device.…”

Section: Introductionmentioning

confidence: 99%

Radar-Based Hand Gesture Recognition Using Spiking Neural Networks

et al. 2021

View full text Add to dashboard Cite

We propose a spiking neural network (SNN) approach for radar-based hand gesture recognition (HGR), using frequency modulated continuous wave (FMCW) millimeter-wave radar. After pre-processing the range-Doppler or micro-Doppler radar signal, we use a signal-to-spike conversion scheme that encodes radar Doppler maps into spike trains. The spike trains are fed into a spiking recurrent neural network, a liquid state machine (LSM). The readout spike signal from the SNN is then used as input for different classifiers for comparison, including logistic regression, random forest, and support vector machine (SVM). Using liquid state machines of less than 1000 neurons, we achieve better than state-of-the-art results on two publicly available reference datasets, reaching over 98% accuracy on 10-fold cross-validation for both data sets.

show abstract

An Efficient Spiking Neural Network for Recognizing Gestures with a DVS Camera on the Loihi Neuromorphic Processor

Cited by 85 publications

References 14 publications

Advancing Neuromorphic Computing With Loihi: A Survey of Results and Outlook

Advancing Neuromorphic Computing With Loihi: A Survey of Results and Outlook

Towards Energy-Efficient and Secure Edge AI: A Cross-Layer Framework

Radar-Based Hand Gesture Recognition Using Spiking Neural Networks

Contact Info

Product

Resources

About