Unsupervised AER Object Recognition Based on Multiscale Spatio-Temporal Features and Spiking Neurons

Abstract

This paper proposes an unsupervised address event representation (AER) object recognition approach. The proposed approach consists of a novel multiscale spatio-temporal feature (MuST) representation of input AER events and a spiking neural network (SNN) using spike-timing-dependent plasticity (STDP) for object recognition with MuST. MuST extracts the features contained in both the spatial and temporal information of AER event flow, and meanwhile forms an informative and compact feature spike representation. We show not only how MuST exploits spikes to convey information more effectively, but also how it benefits the recognition using SNN. The recognition process is performed in an unsupervised manner, which does not need to specify the desired status of every single neuron of SNN, and thus can be flexibly applied in real-world recognition tasks. The experiments are performed on five AER datasets including a new one named GESTURE-DVS. Extensive experimental results show the effectiveness and advantages of this proposed approach.

Spike timing-based unsupervised learning of orientation, disparity, and motion representations in a spiking neural network

“…Lagorce et al [13] presented an interesting unsupervised learning of visual representations, but their implementation is based on a conversion to a rate code and does not use STDP. Liu et al [16] introduced a new "Multiscale Spatio-Temporal Feature" representation and applied it to recognition tasks such as gesture or digit recognition via STDP. Recently, Paredes-Valles et al [24] have presented a multi-stage SNN that is capable of optical flow estimation using STDP-based learning combined with homeostatic mechanisms.…”

Effective AER Object Classification Using Segmented Probability-Maximization Learning in Spiking Neural Networks

“…(Cohen et al 2016) presented an implementation of Synaptic Kernel Inverse Method (SKIM), which is a learning method based on principles of dendritic computation, in order to perform a large-scale AER object classification task. (Liu et al 2019) proposed a multiscale spatio-temporal feature (MuST) representation of AER events and an unsupervised rocognition approach.…”

SpikeSEG: Spiking Segmentation via STDP Saliency Mapping

“…Neuromorphic Vision Sensors have been used successfully in multiple research areas within traditional computer vision and have utilised CNNs for Classification, Motion Estimation and Optical Flow [9]. Though some of the traditional computer vision techniques aim to exploit the sparse event driven nature of the sensor, there has been less focus on this with the CNN approach [30]- [32]. The asynchronous CNN fcYOLE [31] in particular aims to replicate benefits of a SNN by converting an already trained CNN into an asynchronous version with two methods.…”