Asynchronous event-based corner detection and matching

Clady, Xavier; Ieng, Sio-Hoï; Benosman, Ryad

doi:10.1016/j.neunet.2015.02.013

Cited by 102 publications

(100 citation statements)

References 29 publications

Supporting

Mentioning

100

Contrasting

Order By: Relevance

“…The method in [16] does not detect and track user-defined objects but lower-level primitives, such as corner events defined by the intersection of two moving edges, which are obtained by fitting planes in the space-time stream of events.…”

Section: A Event-based Feature Detection and Trackingmentioning

confidence: 99%

Low-latency visual odometry using event-based feature tracks

Kueng

Mueggler

Gallego

et al. 2016

2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

162

159

View full text Add to dashboard Cite

New vision sensors, such as the Dynamic and Active-pixel Vision sensor (DAVIS), incorporate a conventional camera and an event-based sensor in the same pixel array. These sensors have great potential for robotics because they allow us to combine the benefits of conventional cameras with those of eventbased sensors: low latency, high temporal resolution, and high dynamic range. However, new algorithms are required to exploit the sensor characteristics and cope with its unconventional output, which consists of a stream of asynchronous brightness changes (called "events") and synchronous grayscale frames. In this paper, we present a lowlatency visual odometry algorithm for the DAVIS sensor using event-based feature tracks. Features are first detected in the grayscale frames and then tracked asynchronously using the stream of events. The features are then fed to an event-based visual odometry algorithm that tightly interleaves robust pose optimization and probabilistic mapping. We show that our method successfully tracks the 6-DOF motion of the sensor in natural scenes. This is the first work on event-based visual odometry with the DAVIS sensor using feature tracks. Low-Latency Visual Odometry using Event-based Feature Tracks Beat Kueng, Elias Mueggler, Guillermo Gallego and Davide ScaramuzzaAbstract-New vision sensors, such as the Dynamic and Active-pixel Vision sensor (DAVIS), incorporate a conventional camera and an event-based sensor in the same pixel array. These sensors have great potential for robotics because they allow us to combine the benefits of conventional cameras with those of event-based sensors: low latency, high temporal resolution, and high dynamic range. However, new algorithms are required to exploit the sensor characteristics and cope with its unconventional output, which consists of a stream of asynchronous brightness changes (called "events") and synchronous grayscale frames. In this paper, we present a lowlatency visual odometry algorithm for the DAVIS sensor using event-based feature tracks. Features are first detected in the grayscale frames and then tracked asynchronously using the stream of events. The features are then fed to an event-based visual odometry algorithm that tightly interleaves robust pose optimization and probabilistic mapping. We show that our method successfully tracks the 6-DOF motion of the sensor in natural scenes. This is the first work on event-based visual odometry with the DAVIS sensor using feature tracks. MULTIMEDIA MATERIALA video attachment to this work is available on the authors' webpage.

show abstract

Section: A Event-based Feature Detection and Trackingmentioning

confidence: 99%

Low-latency visual odometry using event-based feature tracks

Kueng

Mueggler

Gallego

et al. 2016

2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

162

159

View full text Add to dashboard Cite

show abstract

“…This has been addressed and demonstrated for arbitrary user-defined shapes using event-based adaptions of the Iterative Closest Point (ICP) algorithm [17], gradient descent [18], or Monte-Carlo methods [19] (i.e., by matching events against a uniformly-sampled collection of rotated and scaled versions of the template). Detection and tracking of locally-invariant features, such as corners, directly from event streams has been addressed instead in [20].…”

Section: B Event-based Tracking Literaturementioning

confidence: 99%

“…Intersecting edges create corners, which are "features" that do not suffer from the aperture problem and that have been proven to be optimally trackable in frame-based approaches [10]. Therefore, event-based cameras also allow for the perception of corners, as shown in [20]. We exploit these observations to extract and describe features using the DAVIS frames, and then track them using the event stream, as illustrated in Fig.…”

Section: Feature Detection and Tracking With The Davismentioning

confidence: 99%

Feature detection and tracking with the dynamic and active-pixel vision sensor (DAVIS)

Tedaldi

Gallego

Mueggler

et al. 2016

2016 Second International Conference on Event-Based Control, Communication, and Signal Processing (EBCCSP)

View full text Add to dashboard Cite

Abstract-Because standard cameras sample the scene at constant time intervals, they do not provide any information in the blind time between subsequent frames. However, for many high-speed robotic and vision applications, it is crucial to provide high-frequency measurement updates also during this blind time. This can be achieved using a novel vision sensor, called DAVIS, which combines a standard camera and an asynchronous event-based sensor in the same pixel array. The DAVIS encodes the visual content between two subsequent frames by an asynchronous stream of events that convey pixellevel brightness changes at microsecond resolution. We present the first algorithm to detect and track visual features using both the frames and the event data provided by the DAVIS. Features are first detected in the grayscale frames and then tracked asynchronously in the blind time between frames using the stream of events. To best take into account the hybrid characteristics of the DAVIS, features are built based on large, spatial contrast variations (i.e., visual edges), which are the source of most of the events generated by the sensor. An eventbased algorithm is further presented to track the features using an iterative, geometric registration approach. The performance of the proposed method is evaluated on real data acquired by the DAVIS.

show abstract

“…In view of the text, corner [5][6] is an important factor of text. Corner is a very important characteristic of the image which plays the important role on understanding of graphics and analysis.…”

Section: Introductionmentioning

confidence: 99%