Asynchronous Multi-Hypothesis Tracking of Features with Event Cameras

Abstract

With the emergence of event cameras, increasing research effort has been focusing on processing the asynchronous stream of events. With each event encoding a discrete intensity change at a particular pixel, uniquely time-stamped with high accuracy, this sensing information is so fundamentally different to the data provided by traditional frame-based cameras that most of the wellestablished vision algorithms are not applicable. Inspired by the need of effective event-based tracking, this paper addresses the tracking of generic patch features relying solely on events, while exploiting their asynchronicity and hightemporal resolution. The proposed approach outperforms the state-of-the-art in event-based feature tracking on wellestablished event camera datasets, retrieving longer and more accurate feature tracks at higher a frequency. Considering tracking as an optimization problem of matching the current view to a feature template, the proposed method implements a simple and efficient technique that only requires the evaluation of a discrete set of tracking hypotheses.

Continuous-Time Gaussian Process Motion-Compensation for Event-vision Pattern Tracking with Distance Fields

“…While most of the literature on this topic is focused on corner-events, their detection [33,34,35,36,37,38,39] and tracking [40,41,42,43,44], corner-events are still to be successfully applied to VO due to their limited reliability. Nonetheless, emerging event-driven pattern-tracking techniques as the ones described in [45,46] have found their way to VO systems as the ones described in [47,48]. Another promising approach is the use of event-based line features [49,50,51,52] which exploits the natural sensitivity of event cameras to edges and for which a significant number of event-based VO systems have been proposed in recent years [53,54,55].…”

“…The proposed method includes the use of a refined-on-the-fly template Ē T that is augmented as new batches are co-registered. The template is built upon an image-like histogram of the accumulation of all the motion-compensated events that are spatiotemporally close to the track similar to [45,46]. By binarising and skeletonising it using morphological operations [64] as shown in Fig.…”

Object Tracking with a Fusion of Event-Based Camera and Frame-Based Camera

“…[7,11] use the deep neural network, and make use of a self-supervised training method to solve the problem of the lack of dataset. There are also several object detection and feature tracking algorithm [3,12,13] for event data that has been developed. However, despite the advantages that the event camera has, at the current time the frame image is still more robust in most scenarios.…”

Event-Based Feature Tracking in Continuous Time with Sliding Window Optimization

“…The work by Ignacio et al [1] proposes an event-by-event tracking approach which models different hypotheses per tracked feature. While this work tracks features at a very high rate of up to 12500 events per second, it is still formulated in discrete time and thus does not allow for simple derivative calculations of the trajectory, which can be useful in some applications.…”