Event-based 3D SLAM with a depth-augmented dynamic vision sensor

Weikersdorfer, David; Adrian, David B.; Cremers, Daniel; Conradt, Jörg

doi:10.1109/icra.2014.6906882

Cited by 118 publications

(78 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the system design was limited to planar motions (i.e., 3-DOF) and planar scenes parallel to the plane of motion consisting of artificial B&W line patterns. The method was extended to 3D in [8] but relied on an external RGB-D sensor attached to the event camera for depth estimation. The depth sensor introduces bottlenecks, which deprives the system of the low latency and high-speed advantages of event cameras.…”

Section: Contributionmentioning

confidence: 99%

EVO: A Geometric Approach to Event-Based 6-DOF Parallel Tracking and Mapping in Real Time

Rebecq

Horstschaefer

Gallego

et al. 2017

IEEE Robot. Autom. Lett.

307

232

View full text Add to dashboard Cite

Abstract-We present EVO, an Event-based Visual Odometry algorithm. Our algorithm successfully leverages the outstanding properties of event cameras to track fast camera motions while recovering a semi-dense 3D map of the environment. The implementation runs in real-time on a standard CPU and outputs up to several hundred pose estimates per second. Due to the nature of event cameras, our algorithm is unaffected by motion blur and operates very well in challenging, high dynamic range conditions with strong illumination changes. To achieve this, we combine a novel, event-based tracking approach based on image-to-model alignment with a recent event-based 3D reconstruction algorithm in a parallel fashion. Additionally, we show that the output of our pipeline can be used to reconstruct intensity images from the binary event stream, though our algorithm does not require such intensity information. We believe that this work makes significant progress in SLAM by unlocking the potential of event cameras. This allows us to tackle challenging scenarios that are currently inaccessible to standard cameras.

show abstract

Section: Contributionmentioning

confidence: 99%

EVO: A Geometric Approach to Event-Based 6-DOF Parallel Tracking and Mapping in Real Time

Rebecq

Horstschaefer

Gallego

et al. 2017

IEEE Robot. Autom. Lett.

307

232

View full text Add to dashboard Cite

show abstract

“…In the experiments, they used an upward-looking DVS mounted on a ground robot moving at low speed. The method was extended in [7] to a 3D SLAM system that requires an RGB-D sensor operating in parallel with the DVS.…”

Section: B Event-based Motion Estimationmentioning

confidence: 99%

“…These sensors constitute a paradigm shift since they operate asynchronously, transmitting only the information conveyed by brightness changes in the scene ("events"), at the time they occur with microsecond resolution. Event-driven algorithms have been developed to provide initial solutions to some robotics problems such as pose tracking [3], [4], visual odometry [5], Simultaneous Localization and Mapping (SLAM) [6], [7]. However, some of these approaches used additional sensors, such as depth sensors [5], [7], or were developed for high-contrast scenes [3], [4], [5].…”

Section: Introductionmentioning

confidence: 99%

Low-latency visual odometry using event-based feature tracks

Kueng

Mueggler

Gallego

et al. 2016

2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

162

159

View full text Add to dashboard Cite

New vision sensors, such as the Dynamic and Active-pixel Vision sensor (DAVIS), incorporate a conventional camera and an event-based sensor in the same pixel array. These sensors have great potential for robotics because they allow us to combine the benefits of conventional cameras with those of eventbased sensors: low latency, high temporal resolution, and high dynamic range. However, new algorithms are required to exploit the sensor characteristics and cope with its unconventional output, which consists of a stream of asynchronous brightness changes (called "events") and synchronous grayscale frames. In this paper, we present a lowlatency visual odometry algorithm for the DAVIS sensor using event-based feature tracks. Features are first detected in the grayscale frames and then tracked asynchronously using the stream of events. The features are then fed to an event-based visual odometry algorithm that tightly interleaves robust pose optimization and probabilistic mapping. We show that our method successfully tracks the 6-DOF motion of the sensor in natural scenes. This is the first work on event-based visual odometry with the DAVIS sensor using feature tracks. Low-Latency Visual Odometry using Event-based Feature Tracks Beat Kueng, Elias Mueggler, Guillermo Gallego and Davide ScaramuzzaAbstract-New vision sensors, such as the Dynamic and Active-pixel Vision sensor (DAVIS), incorporate a conventional camera and an event-based sensor in the same pixel array. These sensors have great potential for robotics because they allow us to combine the benefits of conventional cameras with those of event-based sensors: low latency, high temporal resolution, and high dynamic range. However, new algorithms are required to exploit the sensor characteristics and cope with its unconventional output, which consists of a stream of asynchronous brightness changes (called "events") and synchronous grayscale frames. In this paper, we present a lowlatency visual odometry algorithm for the DAVIS sensor using event-based feature tracks. Features are first detected in the grayscale frames and then tracked asynchronously using the stream of events. The features are then fed to an event-based visual odometry algorithm that tightly interleaves robust pose optimization and probabilistic mapping. We show that our method successfully tracks the 6-DOF motion of the sensor in natural scenes. This is the first work on event-based visual odometry with the DAVIS sensor using feature tracks. MULTIMEDIA MATERIALA video attachment to this work is available on the authors' webpage.

show abstract

“…As another example, in the context of SLAM, the DVS was combined with a frame-based RGB-D camera in [30]. The algorithm used a modified particle filter for tracking the current position and orientation of the sensor while at the same time incrementally creating a probabilistic voxel grid map of the previously unknown environment.…”

Section: Related Work: Ego-motion Estimation With Event-based VImentioning

confidence: 99%

Continuous-Time Trajectory Estimation for Event-based Vision Sensors

Mueggler¹,

Gallego²,

Scaramuzza³

2015

Robotics: Science and Systems XI

View full text Add to dashboard Cite

Abstract-Event-based vision sensors, such as the Dynamic Vision Sensor (DVS), do not output a sequence of video frames like standard cameras, but a stream of asynchronous events. An event is triggered when a pixel detects a change of brightness in the scene. An event contains the location, sign, and precise timestamp of the change. The high dynamic range and temporal resolution of the DVS, which is in the order of micro-seconds, make this a very promising sensor for high-speed applications, such as robotics and wearable computing. However, due to the fundamentally different structure of the sensor's output, new algorithms that exploit the high temporal resolution and the asynchronous nature of the sensor are required. In this paper, we address ego-motion estimation for an event-based vision sensor using a continuous-time framework to directly integrate the information conveyed by the sensor. The DVS pose trajectory is approximated by a smooth curve in the space of rigid-body motions using cubic splines and it is optimized according to the observed events. We evaluate our method using datasets acquired from sensor-in-the-loop simulations and onboard a quadrotor performing flips. The results are compared to the ground truth, showing the good performance of the proposed technique.

show abstract

Event-based 3D SLAM with a depth-augmented dynamic vision sensor

Cited by 118 publications

References 12 publications

EVO: A Geometric Approach to Event-Based 6-DOF Parallel Tracking and Mapping in Real Time

EVO: A Geometric Approach to Event-Based 6-DOF Parallel Tracking and Mapping in Real Time

Low-latency visual odometry using event-based feature tracks

Continuous-Time Trajectory Estimation for Event-based Vision Sensors

Contact Info

Product

Resources

About