Event-driven embodied system for feature extraction and object recognition in robotic applications

Wiesmann, Georg; Schraml, Stephan; Litzenberger, M.; Belbachir, Ahmed Nabil; Hofstätter, Michael; Bartolozzi, Chiara

doi:10.1109/cvprw.2012.6238898

Cited by 22 publications

(22 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Event cameras have been used for object tracking [Delbruck and Lichtsteiner, 2007, Drazen et al, 2011, Delbruck and Lang, 2013, surveillance and monitoring [Litzenberger et al, 2006, Piatkowska et al, 2012, object recognition [Wiesmann et al, 2012, Orchard et al, 2015, Lagorce et al, 2016 and gesture control [Lee et al, 2014]. They have also been used for stereo depth estimation [Rogister et al, 2012, Piatkowska et al, 2013 (see also related work in Section 3), 3D panoramic imaging [Schraml et al, 2015], structured light 3D scanning Fig.…”

Section: Event Cameras and Applicationsmentioning

confidence: 99%

EMVS: Event-Based Multi-View Stereo—3D Reconstruction with an Event Camera in Real-Time

et al. 2017

View full text Add to dashboard Cite

Event cameras are bio-inspired vision sensors that output pixel-level brightness changes instead of standard intensity frames. They offer significant advantages over standard cameras, namely a very high dynamic range, no motion blur, and a latency in the order of microseconds. However, because the output is composed of a sequence of asynchronous events rather than actual intensity images, traditional vision algorithms cannot be applied, so that a paradigm shift is needed. We introduce the problem of eventbased multi-view stereo (EMVS) for event cameras and propose a solution to it. Unlike traditional MVS methods, which address the problem of estimating dense 3D structure from a set of known viewpoints, EMVS estimates semi-dense 3D structure from an event camera with known trajectory. Our EMVS solution elegantly exploits two inherent properties of an event camera: (1) its ability to respond to scene edges-which naturally provide semi-dense geometric information without any pre-processing operationand (2) the fact that it provides continuous measurements as the sensor moves. Despite its simplicity (it can be implemented in a few lines of code), our algorithm is able to produce accurate, semi-dense depth maps, without requiring any explicit data association or intensity estimation. We successfully validate our method on both synthetic and real data. Our method is computationally very efficient and runs in real-time on a CPU. Abstract Event cameras are bio-inspired vision sensors that output pixel-level brightness changes instead of standard intensity frames. They offer significant advantages over standard cameras, namely a very high dynamic range, no motion blur, and a latency in the order of microseconds. However, because the output is composed of a sequence of asynchronous events rather than actual intensity images, traditional vision algorithms cannot be applied, so that a paradigm shift is needed. We introduce the problem of Event-based Multi-View Stereo (EMVS) for event cameras and propose a solution to it. Unlike traditional MVS methods, which address the problem of estimating dense 3D structure from a set of known viewpoints, EMVS estimates semidense 3D structure from an event camera with known trajectory. Our EMVS solution elegantly exploits two inherent properties of an event camera: (i) its ability to respond to scene edges-which naturally provide semi-dense geometric information without any preprocessing operation-and (ii) the fact that it provides continuous measurements as the sensor moves. Despite its simplicity (it can be implemented in a few lines of code), our algorithm is able to produce accurate, semidense depth maps, without requiring any explicit data association or intensity estimation. We successfully validate our method on both synthetic and real data. Our method is computationally very efficient and runs in real-time on a CPU.

show abstract

Section: Event Cameras and Applicationsmentioning

confidence: 99%

EMVS: Event-Based Multi-View Stereo—3D Reconstruction with an Event Camera in Real-Time

et al. 2017

View full text Add to dashboard Cite

show abstract

“…The time resolution is in the order of µs. Due to the sparse nature of the events, the amount of data that has to be transferred from the camera to the computer is very low, making it an energy efficient alternative to standard CMOS cameras for the tracking of very quick movement [8,27]. While it is appealing that the megabytes per second of data produced by a digital camera can be compressed to an asynchronous stream of events, these events can not be used directly in computer vision algorithms that operate on a frame basis.…”

Section: Introductionmentioning

confidence: 99%

Real-Time Intensity-Image Reconstruction for Event Cameras Using Manifold Regularisation

2018

View full text Add to dashboard Cite

Event cameras or neuromorphic cameras mimic the human perception system as they measure the per-pixel intensity change rather than the actual intensity level. In contrast to traditional cameras, such cameras capture new information about the scene at MHz frequency in the form of sparse events. The high temporal resolution comes at the cost of losing the familiar per-pixel intensity information. In this work we propose a variational model that accurately models the behaviour of event cameras, enabling reconstruction of intensity images with arbitrary frame rate in real-time. Our method is formulated on a per-event-basis, where we explicitly incorporate information about the asynchronous nature of events via an event manifold induced by the relative timestamps of events. In our experiments we verify that solving the variational model on the manifold produces high-quality images without explicitly estimating optical flow.

show abstract

“…The time resolution is in the order of µs and the bandwidth of commercially available cameras is up to 1.000.000 events/s. Due to the sparse nature of the events, the amount of data that has to be transferred from the camera to the computer is very low, making it an energy efficient alternative to standard CMOS cameras for the tracking of very quick movements [5,27]. Due to the asynchronous nature of events, computer vision algorithms that operate on a frame basis cannot be readily applied.…”

Section: Introductionmentioning

confidence: 99%

Real-time panoramic tracking for event cameras

Reinbacher

Munda

Pock

2017

2017 IEEE International Conference on Computational Photography (ICCP)

View full text Add to dashboard Cite

Event cameras are a paradigm shift in camera technology. Instead of full frames, the sensor captures a sparse set of events caused by intensity changes. Since only the changes are transferred, those cameras are able to capture quick movements of objects in the scene or of the camera itself. In this work we propose a novel method to perform camera tracking of event cameras in a panoramic setting with three degrees of freedom. We propose a direct camera tracking formulation, similar to state-of-the-art in visual odometry. We show that the minimal information needed for simultaneous tracking and mapping is the spatial position of events, without using the appearance of the imaged scene point. We verify the robustness to fast camera movements and dynamic objects in the scene on a recently proposed dataset [18] and self-recorded sequences.

show abstract

Event-driven embodied system for feature extraction and object recognition in robotic applications

Cited by 22 publications

References 17 publications

EMVS: Event-Based Multi-View Stereo—3D Reconstruction with an Event Camera in Real-Time

EMVS: Event-Based Multi-View Stereo—3D Reconstruction with an Event Camera in Real-Time

Real-Time Intensity-Image Reconstruction for Event Cameras Using Manifold Regularisation

Real-time panoramic tracking for event cameras

Contact Info

Product

Resources

About