Depth Estimation and Inpainting with an Unconstrained Camera

Abstract

Unrestricted camera motion and the ability to operate over a range of lens parameters are often desirable when using an off-the-shelf camera. Variations in intrinsic and extrinsic parameters induce defocus and pixel motion, both of which relate to scene structure. We propose a depth estimation approach by elegantly coupling the motion and defocus cues. We further advocate a natural extension of our framework for inpainting both depth and image, using the motion cue. Unlike traditional inpainting, our approach also considers defocus blur. This ensures that the image inpainting is coherent with respect to defocus. We use the belief propagation method in our estimation approach, which also handles occlusions and uses the color image segmentation cue.

Towards Unrestrained Depth Inference with Coherent Occlusion Filling

“…In this paper (1) we extend the inpainting section by also considering the case of removing occluders present in the scene. We note that this is a more general problem than the one considered in Bhavsar and Rajagopalan (2010). (2) We also include a discussion on special cases of our general framework and bring out a couple of non-conventional and important special camera configurations arising out of our general framework.…”

“…The work in Bhavsar and Rajagopalan (2009) reports only depth estimation for camera translation (albeit along all the 3 axes) and with aperture variation. In Bhavsar and Rajagopalan (2010), we consider depth estimation under general camera motion and parameter variation, but inpainting only static missing regions (e.g. due to sensor damage).…”