Towards event camera signal recognition using a lightweight spiking neural network

Abstract

Event cameras are event-driven bio-inspired sensors, owing to the following advantages: high temporal resolution, high dynamic range, low power, and high imaging speed. Artificial neural networks (ANNs) cannot directly process their output spike signal. Spiking neural networks (SNNs) are neuromorphic computing methods with a high temporal resolution and are event-driven, which fits well with event cameras. However, deep SNNs require large storage space and neuronal computing resources, which limits their deployment in mobile edge computing. Based on the feature dimension mapping, this paper proposes a lightweight SNN for embedded devices to reduce storage requirements, improve efficiency, and enhance network performance. First, by analyzing the relationship between the parameter number and the fitting function of the network, the SNN parameter bottleneck is clear. Subsequently, a general lightweight feature extraction structure, SpikeFire, is proposed based on a lowdimensional feature extraction-fusion strategy, which significantly reduces network parameters on the premise that the basic properties of the receptive field and feature dimension are the same as those of a spiking convolution. Additionally, the module simulates the complex connection of the brain and adopts the skip connection, which not only increases the extraction of multi-scale information but also contributes to deep network optimization. Finally, the lightweight network is deployed in embedded hardware to build an event-driven imaging recognition system. Experiments show that our method can reduce the parameter number and improve the running speed without weakening the performance in public, self-built real scenes, and extreme imaging scene datasets.