In Defense of 3D-Label Stereo

Abstract

It is commonly believed that higher order smoothness should be modeled using higher order interactions. For example, 2nd order derivatives for deformable (active) contours are represented by triple cliques. Similarly, the 2nd order regularization methods in stereo predominantly use MRF models with scalar (1D) disparity labels and triple clique interactions. In this paper we advocate a largely overlooked alternative approach to stereo where 2nd order surface smoothness is represented by pairwise interactions with 3D-labels, e.g. tangent planes. This general paradigm has been criticized due to perceived computational complexity of optimization in higher-dimensional label space. Contrary to popular beliefs, we demonstrate that representing 2nd order surface smoothness with 3D labels leads to simpler optimization problems with (nearly) submodular pairwise interactions. Our theoretical and experimental results demonstrate advantages over state-of-the-art methods for 2nd order smoothness stereo. 1

Patchwork Stereo: Scalable, Structure-Aware 3D Reconstruction in Man-Made Environments

“…The situation is even worse with voxel-based approaches [12,30]. Pixel-based stereo techniques, which build disparity maps, have seen a tremendous increase in performance since early approaches [16] and their later extensions using second order smoothness priors [40,27], color models [2] or semantic classification [18]. This category of approaches has been well established for narrowbaseline stereo problems as reported in the Middlebury challenge [32], but it scales poorly in image number and image size; besides, it is sensitive to wider baselines.…”

Divergence Prior and Vessel-Tree Reconstruction

“…Curvature, a second-order smoothness term, is a natural regularizer for thin structures. In general, curvature was studied for image segmentation [24,27,25,5,13,23,20,17], for stereo or multi-view-reconstruction [16,22,30], connectivity measures in analysis of diffusion MRI [19], for tubular structures extraction [17], for inpainting [2,6] and edge completion [11,29,1].…”

Plane labeling trinocular stereo matching with baseline recovery

“…This type of approach is known as scene flow estimation. Using disparity planes to estimate sub-pixel stereo disparity has been previously used in [9,20,4,3,13,7,15,14,21]. These algorithms can be classified in two categories: fixed plane inference (FPI) and dynamic plane inference (DPI).…”