High-Definition 3D Reconstruction in Real-Time from a Moving Depth Sensor

Abstract

-This paper proposes a method to reconstruct 3D point clouds of a static scene in real-time by a moving depth sensor. Based on a base-line 3D reconstruction algorithm which fuses depth maps from the moving camera, a depth map enhancement module is embedded in the depth fusion stage to improve the level of details of reconstructed 3D models. Depth enhancements using the Sobel operator and the Laplacian operator are applied for speed and quality considerations. Experiments for real-time reconstruction of 3D scenes, especially the reconstruction of 3D human faces are provided to validate the proposed method. The reconstruction results are inspiring in their high definition compared with the original base-line algorithm.Index Terms -3D reconstruction, Real-time reconstruction, Depth enhancement, High-definition, Depth sensor. I . IntroductionHigh-quality 3D polygonal models reconstructed and rendered from a real world scene are required in different applications, like video-games, movies, virtual reality, ecommerce and other graphics applications. However the generation of a 3D model with precise surface from unorganized point clouds derived from laser scanner data [5] or photogrammetric image measurements [4,6] is a difficult problem. For the reconstruction of a person's face more level of details and capturing time are required to get data, but it feels uncomfortable for people to keep a pose too long; small movements are inevitable.A number of methods have been developed for 3D reconstructions using data from different deceives. There are methods based on triangulations, such as using laser light [9], structured light [12], coded light [13], and image measurements [6,7,8]. There are also methods do not require triangulations and directly estimate surface normal vectors, such as shape form texture [10] and shape from 2D edge gradients [11].The methods based on 3D active sensors (mainly laser scanners) provide the highest quality of reconstruction at present. However, the 3D active sensors are quite expensive, which are mainly for industry applications. Passive imagebased methods using projective geometry [6] are very portable and the sensors are not expensive. But the accuracy of result is low and it cannot satisfy real-time applications.The Kinect Fusion system [1] developed by Microsoft Research uses low-cost depth sensor (the Kinect) and commodity graphics hardware for accurate and real-time 3D reconstruction. Depth data from Kinect is used to track the 3D pose of the sensor and reconstruct 3D models of the physical scene [2]. The advantage of Kinect Fusion is its speed for permitting direct feedback and user interaction. But it is still required to improve the resolution of 3D reconstruction in Kinect Fusion since the hardware performance is generally limited.In this paper, we shall use a depth map enhancement module to improve the quality of the base-line algorithm of Kinect Fusion. We shall apply image enhancement approaches for depth map enhancement. We implement the 3D reconstruction of face model...