Spatio‐temporal geometry fusion for multiple hybrid cameras using moving least squares surfaces

Abstract

Figure 1: a) Combined raw geometries obtained by a calibrated setup of two hybrid color+depth cameras rendered in green and red respectively. b) Result of geometry fusion obtained by our MLS-based approach. c) Textured geometry from a). Note the numerous visual artifacts due to the inaccurate and incomplete geometry, especially near depth discontinuities. d) Our optimized textured geometry from b). AbstractMulti-view reconstruction aims at computing the geometry of a scene observed by a set of cameras. Accurate 3D reconstruction of dynamic scenes is a key component for a large variety of applications, ranging from special effects to telepresence and medical imaging. In this paper we propose a method based on Moving Least Squares surfaces which robustly and efficiently reconstructs dynamic scenes captured by a calibrated set of hybrid color+depth cameras. Our reconstruction provides spatio-temporal consistency and seamlessly fuses color and geometric information. We illustrate our approach on a variety of real sequences and demonstrate that it favorably compares to state-of-the-art methods.

Real-time scene reconstruction and triangle mesh generation using multiple RGB-D cameras

“…Thus, we used both a dataset created by Kuster et al [33] and our own recorded data. Figure 12 shows them in use.…”

“…To increase result stability, Guennebaud and Gross [22] formulate higher-order surface approximation while Fleishman et al [19] and Wang et al [49] add detail-preserving MLS methods. Kuster et al [33] introduce temporally stable MLS for use in dynamic scenes.…”

“…Their memory requirements are dependent on the number of input points and not on a reconstruction volume. Additionally, the process is fully parallelizable, making it an excellent target for GPU acceleration [23,33].…”

“…Kuster et al [33] propose storing points and normals data as two-dimensional arrays similar to range images. In that case, a lookup operation would consist of projecting search points to every camera and retrieving an s  s block of points around projected coordinates, where s is known as window size.…”

“…To achieve temporal stability, we follow Kuster et al [33] who propose extending x to a 4-dimensional vector that contains not only spatial coordinates but also a time value. Every frame received from a camera has a timestamp that is assigned to the fourth coordinate of all points in the frame.…”