Nonlinear disparity mapping for stereoscopic 3D

Abstract

This paper addresses the problem of remapping the disparity range of stereoscopic images and video. Such operations are highly important for a variety of issues arising from the production, live broadcast, and consumption of 3D content. Our work is motivated by the observation that the displayed depth and the resulting 3D viewing experience are dictated by a complex combination of perceptual, technological, and artistic constraints. We first discuss the most important perceptual aspects of stereo vision and their implications for stereoscopic content creation. We then formalize these insights into a set of basic disparity mapping operators. These operators enable us to control and retarget the depth of a stereoscopic scene in a nonlinear and locally adaptive fashion. To implement our operators, we propose a new strategy based on stereoscopic warping of the input video streams. From a sparse set of stereo correspondences, our algorithm computes disparity and image-based saliency estimates, and uses them to compute a deformation of the input views so as to meet the target disparities. Our approach represents a practical solution for actual stereo production and display that does not require camera calibration, accurate dense depth maps, occlusion handling, or inpainting. We demonstrate the performance and versatility of our method using examples from live action post-production, 3D display size adaptation, and live broadcast. An additional user study and ground truth comparison further provide evidence for the quality and practical relevance of the presented work.

Light field image editing by 4D patch synthesis

“…Stereoscopic image inpainting techniques have been proposed by Wang et al [6] and Morse et al [7]. Warping-based methods have been used frequently for stereoscopic image editing, such as disparity remapping [8] and retargeting [9,10]. On the other hand, patch-based methods have not been used until very recently when Luo et al extended Patch-Match to stereoscopic image editing [11].…”

High-Quality Depth Estimation Using an Exemplar 3D Model for Stereo Conversion

“…Similar to disparity mapping techniques [25] [26], the ground-aware depth adaptation adjusts a depth mapping function with the constraints provided by the contact points. For smooth adjustment, we employ a Laplacian operator.…”

Correspondence and Depth-Image Based Rendering a Hybrid Approach for Free-Viewpoint Video

“…By using precisely triggered lighting and capturing devices, relighting can be incorporated into light-field based free-viewpoint video [11]. For small viewpoint alterations, (dis)occlusion handling can be elegantly circumvented by nonlinear disparity mapping [20]. Correspondence and depth-image based rendering.…”