Floating Textures

Abstract

We present a novel multi-view, projective texture mapping technique. While previous multi-view texturing approaches lead to blurring and ghosting artefacts if 3D geometry and/or camera calibration are imprecise, we propose a texturing algorithm that warps ("floats") projected textures during run-time to preserve crisp, detailed texture appearance. Our GPU implementation achieves interactive to real-time frame rates. The method is very generally applicable and can be used in combination with many image-based rendering methods or projective texturing applications. By using Floating Textures in conjunction with, e.g., visual hull rendering, light field rendering, or free-viewpoint video, improved rendering results are obtained from fewer input images, less accurately calibrated cameras, and coarser 3D geometry proxies.

Towards Plenoptic Raumzeit Reconstruction

“…Our first presented method, the Floating Textures [1], deal with the weaknesses in the classic technique of reconstructing a proxy geometry for discrete timesteps and retexturing it with the input images. To deal with geometric uncertainties, camera calibration errors and visibility problems, Floating Textures proposes to warp the input images in the output domain and reweight the warping and blending paramters based on their angular and occlusion distance.…”

“…To compute the optical flow between two images efficient GPU implementations are needed [1,38]. Even though this processing step is computationally expensive and takes approximately 90% of the rendering time, interactive to real-time speedups are possible with modern GPUs.…”

“…After reviewing related work in the next section we will continue with our Floating Textures [1] in Section 3 as an example for high quality free-viewpoint video with discrete time steps. For each discrete time step a geometry of the scene is reconstructed and textured by multiview projective texture mapping, as this process is error-prone we present a warping-based refinement to correct for the resulting artifacts.…”

Free-Viewpoint Video for TV Sport Production

“…Novel viewpoints are then rendered by texture mapping the reconstructed surface. Other multiple camera studio system with small numbers of cameras (6)(7)(8)(9)(10)(11)(12) have used the visual-hull from multiple view silhouettes [31,35] and photo-hull [47]. Real-time free-viewpoint video with interactive viewpoint control has also been demonstrated [13,31].…”

“…This approach refines an initial visual-hull reconstruction to obtain a surface which gives accurate alignment between widely spaced views. As an alternative to refinement of geometry reconstruction direct texture correspondence and interpolation using optic-flow tech-niques has been used to correct misalignment due to errors in geometry and achieve high-quality rendering on relatively low-resolution models [10] This approach could be advantageous in the presence of camera calibration error as it avoids the requirement for a single surface reconstruction which is consistent between all views.…”