A Motion-Based Feature for Event-Based Pattern Recognition

Abstract

This paper introduces an event-based luminance-free feature from the output of asynchronous event-based neuromorphic retinas. The feature consists in mapping the distribution of the optical flow along the contours of the moving objects in the visual scene into a matrix. Asynchronous event-based neuromorphic retinas are composed of autonomous pixels, each of them asynchronously generating “spiking” events that encode relative changes in pixels' illumination at high temporal resolutions. The optical flow is computed at each event, and is integrated locally or globally in a speed and direction coordinate frame based grid, using speed-tuned temporal kernels. The latter ensures that the resulting feature equitably represents the distribution of the normal motion along the current moving edges, whatever their respective dynamics. The usefulness and the generality of the proposed feature are demonstrated in pattern recognition applications: local corner detection and global gesture recognition.

HATS: Histograms of Averaged Time Surfaces for Robust Event-Based Object Classification

“…Recently, [12] introduced a feature descriptor based on local distributions of optical flow and applied it to corner detection and gesture recognition. It is inspired by its framebased counterpart [10], but in [12] the algorithm for computing the optical flow relies on the temporal information carried by the events. One limitation of [12] is that the quality of the descriptor strongly depends on the quality of the flow.…”

“…It is inspired by its framebased counterpart [10], but in [12] the algorithm for computing the optical flow relies on the temporal information carried by the events. One limitation of [12] is that the quality of the descriptor strongly depends on the quality of the flow. As a consequence, it loses accuracy in presence of noise or poorly contrasted edges.…”

ACE: An Efficient Asynchronous Corner Tracker for Event Cameras

“…More complex data association schemes have been proposed, but mainly aimed to high-level tasks, such as pattern recognition [8,9,19,5], where prior training to a model is required. Therefore, these methods are not suitable for on-line corner-event tracking, where no a priori knowledge of the scene can be assumed.…”

Asynchronous, Photometric Feature Tracking Using Events and Frames

“…Feature detection and tracking with event cameras is a major research topic [8,9,[12][13][14][15][16][17][18], where the goal is to unlock the capabilities of event cameras and use Fig. 1(a): Comparison of the output of a standard frame-based camera and an event camera when facing a black dot on a rotating disk (figure adapted from [11]).…”

“…Recently, extensions of popular image-based keypoint detectors, such as Harris [19] and FAST [20], have been developed for event cameras [17,18]. Detectors based on the distribution of optical flow [21] for recognition applications have also been proposed for event cameras [16]. Finally, most event-based trackers use binary feature templates, either predefined [13] or built from a set of events [9], to which they align events by means of iterative point-set-based methods, such as iterative closest point (ICP) [22].…”