A Novel HDR Depth Camera for Real-Time 3D 360° Panoramic Vision

Abstract

This paper presents a novel 360° High-Dynamic Range (HDR) camera for real-time 3D 360° panoramic computer vision. The camera consists of (1) a pair of bio-inspired dynamic vision line sensors (1024 pixel each) asynchronously generating events at high temporal resolution with on-chip time stamping (1μs resolution), having a high dynamic range and the sparse visual coding of the information, (2) a high-speed mechanical device rotating at up to 10 revolutions per sec (rps) on which the pair of sensor is mounted and (3) a processing unit for the configuration of the detector chip and transmission of its data through a slip ring and a gigabit Ethernet communication to the user. Within this work, we first present the new camera, its individual components and resulting panoramic edge map. In a second step, we developed a method for reconstructing the intensity images out of the event data generated by the sensors. The algorithm maps the recorded panoramic views into graylevel images by using a transform coefficient. In the last part of this work, anaglyph representation and 3D reconstruction results out of the stereo images are shown. The experimental results show the capabilities of the new camera to generate 10 x 3D panoramic views per second in real-time at an image resolution of 5000x1024 pixel and intra-scene dynamic range of more than 120 dB under natural illuminations. The camera potential for 360° depth imaging and mobile computer vision is briefly highlighted.

Event-Based Vision: A Survey

“…Each pixel of the panoramic image used a Kalman filter to estimate the brightness gradient (based on (4)), which was then integrated using Poisson reconstruction to yield absolute brightness. The method in [203] exploited the constrained motion of a platform rotating around a single axis to reconstruct images that were then used for stereo depth estimation. Motion restrictions were then replaced by regularizing assumptions to enable image reconstruction for generic motions and scenes [101].…”

3-D Imaging Systems for Agricultural Applications—A Review

“…The previously mentioned sensors rely on triangulation techniques that were not included in the classification proposed by Schwarte et al [ 14 ]. The Austrian Institute of Technology [ 134 ] has recently developed a dynamic stereo vision camera that continuously rotates to generate a real-time 360° 3-D view. This camera exploits the high sampling rate and low latency capabilities of the dynamic vision sensor (DVS) that only senses changes at a pixel-level, caused by movement, significantly reducing the amount of acquired data.…”

Sliding Window Mapping for Omnidirectional RGB-D Sensors

“…A fusion algorithm of RGB-D and odometry data presented by Reference [17] uses a hand-held Microsoft Kinect for the reconstruction of indoor environments. The work described in Reference [18] is an RGB-D sensor using stereoscopic RGB cameras. The cameras rotate in 10 revolutions per second, allowing a full panoramic view, but only in grayscale.…”