Dynamic stereo vision system for real-time tracking

Schraml, Stephan; Belbachir, Ahmed Nabil; Milošević, Nenad; Schön, Peter

doi:10.1109/iscas.2010.5537289

Cited by 51 publications

(50 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These methods follow a two-step approach: first they solve the event correspondence problem across image planes and then triangulate the location of the 3D point. Events are matched in two ways: either using traditional stereo methods on artificial frames generated by accumulating events over time [Schraml et al, 2010, Kogler et al, 2011a, or exploiting simultaneity and temporal correlations of the events across sensors [Kogler et al, 2011b, Rogister et al, 2012, Lee et al, 2012, Camunas-Mesa et al, 2014.…”

Section: Related Work On Event-based Depth Estimationmentioning

confidence: 99%

EMVS: Event-Based Multi-View Stereo—3D Reconstruction with an Event Camera in Real-Time

et al. 2017

View full text Add to dashboard Cite

Event cameras are bio-inspired vision sensors that output pixel-level brightness changes instead of standard intensity frames. They offer significant advantages over standard cameras, namely a very high dynamic range, no motion blur, and a latency in the order of microseconds. However, because the output is composed of a sequence of asynchronous events rather than actual intensity images, traditional vision algorithms cannot be applied, so that a paradigm shift is needed. We introduce the problem of eventbased multi-view stereo (EMVS) for event cameras and propose a solution to it. Unlike traditional MVS methods, which address the problem of estimating dense 3D structure from a set of known viewpoints, EMVS estimates semi-dense 3D structure from an event camera with known trajectory. Our EMVS solution elegantly exploits two inherent properties of an event camera: (1) its ability to respond to scene edges-which naturally provide semi-dense geometric information without any pre-processing operationand (2) the fact that it provides continuous measurements as the sensor moves. Despite its simplicity (it can be implemented in a few lines of code), our algorithm is able to produce accurate, semi-dense depth maps, without requiring any explicit data association or intensity estimation. We successfully validate our method on both synthetic and real data. Our method is computationally very efficient and runs in real-time on a CPU. Abstract Event cameras are bio-inspired vision sensors that output pixel-level brightness changes instead of standard intensity frames. They offer significant advantages over standard cameras, namely a very high dynamic range, no motion blur, and a latency in the order of microseconds. However, because the output is composed of a sequence of asynchronous events rather than actual intensity images, traditional vision algorithms cannot be applied, so that a paradigm shift is needed. We introduce the problem of Event-based Multi-View Stereo (EMVS) for event cameras and propose a solution to it. Unlike traditional MVS methods, which address the problem of estimating dense 3D structure from a set of known viewpoints, EMVS estimates semidense 3D structure from an event camera with known trajectory. Our EMVS solution elegantly exploits two inherent properties of an event camera: (i) its ability to respond to scene edges-which naturally provide semi-dense geometric information without any preprocessing operation-and (ii) the fact that it provides continuous measurements as the sensor moves. Despite its simplicity (it can be implemented in a few lines of code), our algorithm is able to produce accurate, semidense depth maps, without requiring any explicit data association or intensity estimation. We successfully validate our method on both synthetic and real data. Our method is computationally very efficient and runs in real-time on a CPU.

show abstract

Section: Related Work On Event-based Depth Estimationmentioning

confidence: 99%

EMVS: Event-Based Multi-View Stereo—3D Reconstruction with an Event Camera in Real-Time

et al. 2017

View full text Add to dashboard Cite

show abstract

“…An event-toframe converter was presented in [13] and tested on two conventional stereo-vision algorithms. Another example of DVS event accumulation was shown in [14], where events were accumulated in artificial time slots of 5-50 ms and used in stereo vision for tracking moving objects. In both cases, the event-to-frame conversion was a time-consuming process that introduced some latency and, therefore, the asynchronous data delivery and high temporal resolution of the DVS was not used very efficiently.…”

Section: B Related Workmentioning

confidence: 99%

Lifetime estimation of events from Dynamic Vision Sensors

Mueggler

Förster

Baumli

et al. 2015

2015 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

We propose an algorithm to estimate the "lifetime" of events from retinal cameras, such as a Dynamic Vision Sensor (DVS). Unlike standard CMOS cameras, a DVS only transmits pixel-level brightness changes ("events") at the time they occur with micro-second resolution. Due to its low latency and sparse output, this sensor is very promising for highspeed mobile robotic applications. We develop an algorithm that augments each event with its lifetime, which is computed from the event's velocity on the image plane. The generated stream of augmented events gives a continuous representation of events in time, hence enabling the design of new algorithms that outperform those based on the accumulation of events over fixed, artificially-chosen time intervals. A direct application of this augmented stream is the construction of sharp gradient (edge-like) images at any time instant. We successfully demonstrate our method in different scenarios, including highspeed quadrotor flips, and compare it to standard visualization methods. Lifetime Estimation of Events from Dynamic Vision SensorsElias Mueggler, Christian Forster, Nathan Baumli, Guillermo Gallego and Davide Scaramuzza Abstract-We propose an algorithm to estimate the "lifetime" of events from retinal cameras, such as a Dynamic Vision Sensor (DVS). Unlike standard CMOS cameras, a DVS only transmits pixel-level brightness changes ("events") at the time they occur with micro-second resolution. Due to its low latency and sparse output, this sensor is very promising for highspeed mobile robotic applications. We develop an algorithm that augments each event with its lifetime, which is computed from the event's velocity on the image plane. The generated stream of augmented events gives a continuous representation of events in time, hence enabling the design of new algorithms that outperform those based on the accumulation of events over fixed, artificially-chosen time intervals. A direct application of this augmented stream is the construction of sharp gradient (edge-like) images at any time instant. We successfully demonstrate our method in different scenarios, including highspeed quadrotor flips, and compare it to standard visualization methods.

show abstract

“…These methods follow a two-step approach: first they solve the event correspondence problem across image planes and then triangulate the location of the 3D point. Events are matched in two ways: either using traditional stereo methods on artificial frames generated by accumulating events over time [7,11], or exploiting simultaneity and temporal correlations of the events across sensors [2,6,8,10].…”

Section: Related Work On Event-based Depth Estimationmentioning

confidence: 99%

“…This is the main difference between our method and existing event-based depth estimation methods (Section 1.1). While previous works essentially attempt to estimate depth by first solving the stereo correspondence problem in the image plane (using frames of accumulated events [7,11], temporal correlation of events [2, 6, 8, 10], etc. ), our method works directly in the 3D space.…”

Section: Feature-viewing Rays By Event Back-projectionmentioning

confidence: 99%

EMVS: Event-based Multi-View Stereo

Rebecq¹,

Gallego²,

Scaramuzza³

2016

Procedings of the British Machine Vision Conference 2016

View full text Add to dashboard Cite

Event cameras are bio-inspired vision sensors that output pixel-level brightness changes instead of standard intensity frames. They offer significant advantages over standard cameras, namely a very high dynamic range, no motion blur, and a latency in the order of microseconds. However, because the output is composed of a sequence of asynchronous events rather than actual intensity images, traditional vision algorithms cannot be applied, so that a paradigm shift is needed. We introduce the problem of Eventbased Multi-View Stereo (EMVS) for event cameras and propose a solution to it. Unlike traditional MVS methods, which address the problem of estimating dense 3D structure from a set of known viewpoints, EMVS estimates semi-dense 3D structure from an event camera with known trajectory. Our EMVS solution elegantly exploits two inherent properties of an event camera: (i) its ability to respond to scene edges-which naturally provide semidense geometric information without any pre-processing operationand (ii) the fact that it provides continuous measurements as the sensor moves. Despite its simplicity (it can be implemented in a few lines of code), our algorithm is able to produce accurate, semidense depth maps. We successfully validate our method on both synthetic and real data. Our method is computationally very efficient and runs in real-time on a CPU. Abstract Event cameras are bio-inspired vision sensors that output pixel-level brightness changes instead of standard intensity frames. They offer significant advantages over standard cameras, namely a very high dynamic range, no motion blur, and a latency in the order of microseconds. However, because the output is composed of a sequence of asynchronous events rather than actual intensity images, traditional vision algorithms cannot be applied, so that a paradigm shift is needed. We introduce the problem of Event-based Multi-View Stereo (EMVS) for event cameras and propose a solution to it. Unlike traditional MVS methods, which address the problem of estimating dense 3D structure from a set of known viewpoints, EMVS estimates semi-dense 3D structure from an event camera with known trajectory. Our EMVS solution elegantly exploits two inherent properties of an event camera: (i) its ability to respond to scene edges-which naturally provide semidense geometric information without any pre-processing operation-and (ii) the fact that it provides continuous measurements as the sensor moves. Despite its simplicity (it can be implemented in a few lines of code), our algorithm is able to produce accurate, semidense depth maps. We successfully validate our method on both synthetic and real data. Our method is computationally very efficient and runs in real-time on a CPU. REBECQ, GALLEGO, SCARAMUZZA: EMVS: EVENT-BASED MULTI-VIEW STEREO

show abstract

Dynamic stereo vision system for real-time tracking

Cited by 51 publications

References 7 publications

EMVS: Event-Based Multi-View Stereo—3D Reconstruction with an Event Camera in Real-Time

EMVS: Event-Based Multi-View Stereo—3D Reconstruction with an Event Camera in Real-Time

Lifetime estimation of events from Dynamic Vision Sensors

EMVS: Event-based Multi-View Stereo

Contact Info

Product

Resources

About