Unsupervised Learning of Spatio-Temporal Receptive Fields from an Event-Based Vision Sensor

Abstract

Neuromorphic vision sensors exhibit several advantages compared to conventional frame-based cameras including low latencies, high dynamic range, and low data rates. However, how efficient visual representations can be learned from the output of such sensors in an unsupervised fashion is still an open problem. Here we present a spiking neural network that learns spatio-temporal receptive fields in an unsupervised way from the output of a neuromorphic event-based vision sensor. Learning relies on the combination of spike timing-dependent plasticity with different synaptic delays, the homeostatic regulations of synaptic weights and firing thresholds, and fast inhibition among neurons to decorrelate their responses. Our network develops biologically plausible spatio-temporal receptive fields when trained on real world input and is suited for implementation on neuromorphic hardware.

Spike timing-based unsupervised learning of orientation, disparity, and motion representations in a spiking neural network

“…As the bars are moving from left to right in the visual field, events of specific polarities will always appear in the same order (positive then negative). The learned receptive fields reflect this specific order and resemble Gabor functions describing biological simple cell receptive fields [2].…”

Event Cameras in Automotive Sensing: A Review

“…-Schnider et al [127] investigate adaptive learning techniques for SNNs, aiming to improve the robustness and accuracy of optical flow estimation from event-based data. Algorithmic and Architectural Advancements -Barbier et al [113], Liu et al [114], and Zhuang et al [116] present novel algorithms and network architectures designed to tackle the challenges of event-based optical flow estimation, such as handling high-speed motion blur and improving feature representation.…”

Event Camera Simulator Improvements via Characterized Parameters

“…These include the simulation of complex environments where the control strategy is in the loop (Kaiser et al, 2016 ) and the simulation of colour event-based sensors which are not yet widely available (García et al, 2016 ; Scheerlinck et al, 2019 ). The accuracy of these simulators varies from precise electrical simulations of the pixel (Remy, 2019 ) designed to optimise the sensor's performance, to high-level simulations of well-defined problems like the simulation of a moving bar (Barbier et al, 2020 ). These high-level simulations are often sufficient for understanding and exploring the performance of algorithms, and one of the first simulations of an event-based pixel was implemented to train a robot to follow a line (Kaiser et al, 2016 ).…”