4.1 A 640×480 dynamic vision sensor with a 9µm pixel and 300Meps address-event representation

Bongki, Son; Suh, Yunjae; Kim, Sungho; Jung, Hyunwoo; Kim, Sundo; Shin, Changyong; Park, Keunju; Lee, Kyoobin; Park, Jinman; Woo, JongRoul; Roh, Young Jun; Lee, Hyunku; Wang, Yibing; Ovsiannikov, Ilia; Ryu, Hyunsurk

doi:10.1109/isscc.2017.7870263

Cited by 146 publications

(116 citation statements)

References 4 publications

Supporting

Mentioning

110

Contrasting

Order By: Relevance

“…The following experiment shows that our method is not sensitive to the number of clusters chosen N . We found that N is not a particularly important parameter; if it cho- Figure 6: A 640×480 pixel DVS [37] panning over a slanted plane. Segmentation with 10 optical flow clusters (colored).…”

Section: Sensitivity To the Number Of Clustersmentioning

confidence: 90%

“…Fig. 6 shows that our method also works with a higher resolution (640 × 480 pixels) event-based camera [37]. More experiments are provided in the Appendix.…”

Section: Further Real-world Sequencesmentioning

confidence: 96%

“…Due to the recent development of new, high resolution event-based cameras [37], we show the results of our method on the output of a Samsung DVS Gen3 sensor, with a spatial resolution of 640 × 480 pixels. In this experiment we show the segmentation of several scenes (a textured carpet, some leaves and a temple poster) as the camera moves.…”

Section: C3 Continuum Depth Variation With High-resolution Event-bamentioning

confidence: 99%

See 2 more Smart Citations

Event-Based Motion Segmentation by Motion Compensation

Stoffregen

Gallego

Drummond

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

146

119

View full text Add to dashboard Cite

Figure 1: Our method segments a set of events produced by an event-based camera (Left, with color image of the scene for illustration) into the different moving objects causing them (Right: pedestrian, cyclist and camera's ego-motion, in color). We propose an iterative clustering algorithm (Middle block) that jointly estimates the motion parameters θ and event-cluster membership probabilities P to best explain the scene, yielding motion-compensated event images on all clusters (Right). AbstractIn contrast to traditional cameras, whose pixels have a common exposure time, event-based cameras are novel bio-inspired sensors whose pixels work independently and asynchronously output intensity changes (called "events"), with microsecond resolution. Since events are caused by the apparent motion of objects, event-based cameras sample visual information based on the scene dynamics and are, therefore, a more natural fit than traditional cameras to acquire motion, especially at high speeds, where traditional cameras suffer from motion blur. However, distinguishing between events caused by different moving objects and by the camera's ego-motion is a challenging task. We present the first per-event segmentation method for splitting a scene into independently moving objects. Our method jointly estimates the event-object associations (i.e., segmentation) and the motion parameters of the objects (or the background) by maximization of an objective function, which builds upon recent results on event-based motion-compensation. We provide a thorough evaluation of our method on a public dataset, outperforming the state-of-the-art by as much as 10 %. We also show the first quantitative evaluation of a segmentation algorithm for event cameras, yielding around 90 % accuracy at 4 pixels relative displacement. Supplementary MaterialAccompanying video: https://youtu.be/0q6ap OSBAk. We encourage the reader to view the added experiments and theory in the supplement.

show abstract

Section: Sensitivity To the Number Of Clustersmentioning

confidence: 90%

“…Fig. 6 shows that our method also works with a higher resolution (640 × 480 pixels) event-based camera [37]. More experiments are provided in the Appendix.…”

Section: Further Real-world Sequencesmentioning

confidence: 96%

Section: C3 Continuum Depth Variation With High-resolution Event-bamentioning

confidence: 99%

See 1 more Smart Citation

Event-Based Motion Segmentation by Motion Compensation

Stoffregen

Gallego

Drummond

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

146

119

View full text Add to dashboard Cite

show abstract

“…Since there exists no public event dataset containing fast physical phenomena, we recorded our own. We used the Samsung DVS Gen3 sensor [41], with VGA resolution. We recorded four sequences at daytime under bright sunlight (Fig.…”

Section: High Speed Video Reconstructionmentioning

confidence: 99%

High Speed and High Dynamic Range Video with an Event Camera

Rebecq

Ranftl

Koltun

et al. 2021

IEEE Trans. Pattern Anal. Mach. Intell.

389

339

View full text Add to dashboard Cite

Event cameras are novel sensors that report brightness changes in the form of a stream of asynchronous "events" instead of intensity frames. They offer significant advantages with respect to conventional cameras: high temporal resolution, high dynamic range, and no motion blur. While the stream of events encodes in principle the complete visual signal, the reconstruction of an intensity image from a stream of events is an ill-posed problem in practice. Existing reconstruction approaches are based on hand-crafted priors and strong assumptions about the imaging process as well as the statistics of natural images. In this work we propose to learn to reconstruct intensity images from event streams directly from data instead of relying on any hand-crafted priors. We propose a novel recurrent network to reconstruct videos from a stream of events, and train it on a large amount of simulated event data. During training we propose to use a perceptual loss to encourage reconstructions to follow natural image statistics. We further extend our approach to synthesize color images from color event streams. Our quantitative experiments show that our network surpasses state-of-the-art reconstruction methods by a large margin in terms of image quality (> 20%), while comfortably running in real-time. We show that the network is able to synthesize high framerate videos (> 5,000 frames per second) of high-speed phenomena (e.g. a bullet hitting an object) and is able to provide high dynamic range reconstructions in challenging lighting conditions. As an additional contribution, we demonstrate the effectiveness of our reconstructions as an intermediate representation for event data. We show that off-the-shelf computer vision algorithms can be applied to our reconstructions for tasks such as object classification and visual-inertial odometry and that this strategy consistently outperforms algorithms that were specifically designed for event data. We release the reconstruction code and a pre-trained model to enable further research.

show abstract

“…Increase in the practical application of DVS has driven many companies to get involved in research and development of the sensor. Variants of DVS such as DVS128 (Lichtsteiner et al, ), eDVS, mini‐eDVS, DAVIS240 (240 × 180) (Brandli, Berner, Yang, Liu, & Delbruck, , ), DAVIS346 (346 × 260) and color‐DAVIS (Li et al, ) have been developed by iniVation Inc. Samsung Inc. developed DVS (Son, Suh, et al, ) with lesser pixel size, low power, good event quality, and low data rate. The data to be sent can be programmed.…”

Section: Silicon Retinamentioning

confidence: 99%

Neuromorphic vision: From sensors to event‐based algorithms

Lakshmi

Chakraborty

Thakur

2019

WIREs Data Min & Knowl

View full text Add to dashboard Cite

Regardless of the marvels brought by the conventional frame‐based cameras, they have significant drawbacks due to their redundancy in data and temporal latency. This causes problem in applications where low‐latency transmission and high‐speed processing are mandatory. Proceeding along this line of thought, the neurobiological principles of the biological retina have been adapted to accomplish data sparsity and high dynamic range at the pixel level. These bio‐inspired neuromorphic vision sensors alleviate the more serious bottleneck of data redundancy by responding to changes in illumination rather than to illumination itself. This paper reviews in brief one such representative of neuromorphic sensors, the activity‐driven event‐based vision sensor, which mimics human eyes. Spatio‐temporal encoding of event data permits incorporation of time correlation in addition to spatial correlation in vision processing, which enables more robustness. Henceforth, the conventional vision algorithms have to be reformulated to adapt to this new generation vision sensor data. It involves design of algorithms for sparse, asynchronous, and accurately timed information. Theories and new researches have begun emerging recently in the domain of event‐based vision. The necessity to compile the vision research carried out in this sensor domain has turned out to be considerably more essential. Towards this, this paper reviews the state‐of‐the‐art event‐based vision algorithms by categorizing them into three major vision applications, object detection/recognition, object tracking, localization and mapping. This article is categorized under: Technologies > Machine Learning

show abstract

4.1 A 640×480 dynamic vision sensor with a 9µm pixel and 300Meps address-event representation

Cited by 146 publications

References 4 publications

Event-Based Motion Segmentation by Motion Compensation

Event-Based Motion Segmentation by Motion Compensation

High Speed and High Dynamic Range Video with an Event Camera

Neuromorphic vision: From sensors to event‐based algorithms

Contact Info

Product

Resources

About