Integrating approximate depth data into dense image correspondence estimation

Abstract

High-quality dense image correspondence estimation between two images is an essential prerequisite for many tasks in visual media production, one prominent example being view interpolation. Due to the ill-posed nature of the correspondence estimation problem, errors occur frequently for a number of problematic conditions, among them occlusions, large displacements and low-textured regions. In this paper, we propose to use approximate depth data from low-resolution depth sensors or coarse geometric proxies to guide the high-resolution image correspondence estimation. We counteract the effect of uncertainty in the prior by exploiting the coarse-tofine image pyramid used in our estimation algorithm. Our results show that even with only approximate priors, visual quality improves considerably compared to an unguided algorithm or a pure depth-based interpolation.

A Survey on Time-of-Flight Stereo Fusion

“…They are often based on a line search that is guided by the ToF data. Global methods, [5,31,35,27,44,26,32,51,52,40,45] add the ToF information as an additional data term in a global energy functional is then jointly optimized. While the depth maps obtained are smoother due to the usage of prior information/regularizers, this is at the cost of additional computational resources.…”

“…In this overview, we will further group the global techniques depending the framework that was chosen for optimization. While [31,35,27,44,45] employ different graphical models for inference, [51,32] formulate the problem in a variational framework. The last sub-group of the global methods [5,26,52,40] contains those which use other non-local optimization strategies.…”

“…[46] applies a hierarchical stereo matching algorithm directly on the remapped ToF depth data. The reprojected depth is input into the 4th coursest level of a hierarchical stereo matching algorithm by van Meerbergen et al [53] (see also [51]). Hahne et al 2009 [48].…”

“…We start with a brief overview of a general variational framework, before discussing in detail two recently proposed ToF stereo fusions approaches [32,51], [60], local and variational fusion. We observe that the local approach gives a rough estimate of the disparity, which still compares favorably with SGM.…”

“…Ruhl et al 2012 [51]. The authors consider a fusion system, which focuses on settings with unsynchronized cameras.…”