A Cortically-inspired Architecture for Event-based Visual Motion Processing: From Design Principles to Real-world Applications

Abstract

We developed and tested the architecture of a bioinspired Spiking Neural Network for motion estimation. The computation performed by the retina is emulated by the neuromorphic event-based image sensor DAVIS346 which constitutes the input of our network. We obtained neurons highly tuned to spatial frequency and orientation of the stimulus through a combination of feed-forward excitatory connections modeled as an elongated Gaussian kernel and recurrent inhibitory connections from two clusters of neurons within the same cortical layers. Sums over adjacent nodes weighted by time-variable synapses are used to attain Gabor-like spatio-temporal V1 receptive fields with selectivity to the stimulus' motion. In order to gain the invariance to the stimulus phase, the two polarities of the events provided by the neuromorphic sensor were exploited, which allowed us to build two pairs of quadrature filters from which we obtain Motion Energy detectors as described in [2]. Finally, a decoding stage allows us to compute optic flow from the Motion Detector layers. We tested the approach proposed with both synthetic and natural stimuli.

Egomotion from event-based SNN optical flow

“…Previous works on SNN-based dense optical flow estimation have used bio-inspired Gabor filters [23,25] or unsupervised learning with STDP to learn these filters [4,24]. However, these methods suffer from the common problem of spatio-temporal filter methods, which can only be deployed for test data similar to that used for training.…”