Towards Unrestrained Depth Inference with Coherent Occlusion Filling

Abstract

Traditional depth estimation methods typically exploit the effect of either the variations in internal parameters such as aperture and focus (as in depth from defocus), or variations in extrinsic parameters such as position and orientation of the camera (as in stereo). When operating offthe-shelf (OTS) cameras in a general setting, these parameters influence the depth of field (DOF) and field of view (FOV). While DOF mandates one to deal with defocus blur, a larger FOV necessitates camera motion during image acquisition. As a result, for unfettered operation of an OTS camera, it becomes inevitable to account for pixel motion as well as optical defocus blur in the captured images. We propose a depth estimation framework using calibrated images captured under general camera motion and lens parameter variations. Our formulation seeks to generalize the constrained areas of stereo and shape from defocus (SFD)/focus (SFF) by handling, in tandem, various effects such as focus variation, zoom, parallax and stereo occlusions, all under one roof. One of the associated challenges in such an unrestrained scenario is the problem of removing user-defined foreground occluders in the reference depth map and image (termed inpainting of depth and image). Inpainting is achieved by exploiting the cue from motion parallax to discover (in other images) the correspondence/color information missing in the reference image. Moreover, considering the fact that the observations could be differently blurred, it is important to ensure that the degree of defocus in the missing regions (in the reference image) is coherent with the local neighbours (defocus inpainting).

Disparity Map Generation Based on Trapezoidal Camera Architecture for Multi-View Video

“…Let k be the same ratio by which and divide the non-parallel sides. The mathematical analysis provided by [12] shows that the stereoscopic distance can be obtained as (12b) The dependency of the diagonals on the sides and β can be formulated for an efficient trapezoid (all sides are equal) as (13) (14) (15) (16) The relationship in (16) states that for an efficient trapezoid, the sum of the squares of the diagonal is directly proportional to the square of the side length . The constant of proportionality equals 6.…”

Untitled

“…Let k be the same ratio by which and divide the non-parallel sides. The mathematical analysis provided by [12] shows that the stereoscopic distance can be obtained as…”

“…This includes the issue of camera density, the reduction in the number of physical cameras, image quality, synchronization [9], depth estimation and occlusion [10], [11]. Fairly recently, in [12], other issues such as visual attention has been considered as an important aspect of perception and its understanding is therefore necessary in the creation of 3D content. Reference [13] has targeted an important discussion at remapping the disparity range of stereoscopic images and video.…”

Blur-Aware Disparity Estimation from Defocus Stereo Images

“…However, this method requires two focal stacks from each view, and the camera parameters should be known in advance. This method is generalized in [1] to couple the blur, motion, and depth using a calibrated setup. Several techniques estimate the point spread function without calibrating the camera parameters for depth inference [15,21].…”