Scene flow estimation by growing correspondence seeds

Abstract

A simple seed growing algorithm for estimating scene flow in a stereo setup is presented. Two calibrated and synchronized cameras observe a scene and output a sequence of image pairs. The algorithm simultaneously computes a disparity map between the image pairs and optical flow maps between consecutive images. This, together with calibration data, is an equivalent representation of the 3D scene flow, i.e. a 3D velocity vector is associated with each reconstructed point. The proposed method starts from correspondence seeds and propagates these correspondences to their neighborhood. It is accurate for complex scenes with large motions and produces temporallycoherent stereo disparity and optical flow results. The algorithm is fast due to inherent search space reduction. An explicit comparison with recent methods of spatiotemporal stereo and variational optical and scene flow is provided.

Efficient and Scalable 4th-Order Match Propagation

“…We are purely based on features, like [15], rather than photometric similarity. We do not require images (pixels) after feature detection, unlike [13,14,[26][27][28], nor a regular flow of images [27] or epipolarly rectified image pairs [27]. All these characteristics are crucial for robustness and precision in difficult settings.…”

“…In [13], a region point only defines a single affinity to select admissible candidates, while in [15], growing is via agglomerative clustering. Our propagation is isotropic, image-order insensitive, scale-invariant and adapts to varying detection density like [15], contrary to fixed-size grid in model image [14], fixed-size pixel neighborhood [13,26,27] or reference image [28]. We are purely based on features, like [15], rather than photometric similarity.…”

“…Although it uses known ideas for matching under affinity constraint [18], it includes original ingredients and, as a whole, provides a unique blend. Our propagation is based on local affinities like [13][14][15], not pixel adjacency [26,27], flow [26] or similitudes [28]. Our affinities are computed from match triples (any kind of feature points, possibly in combination), not necessarily affine correspondences [14,15], 2nd moment matrix plus gradient orientation [13], or patch transformations [26,28].…”

High-resolution depth maps based on TOF-stereo fusion

“…We used the Middlebury-2006 dataset 6 . On purpose, we selected three challenging scenes with weakly textured surfaces: Lampshade-1, Monopoly, Plastic.…”

“…A practical tradeoff between the local and the global methods in stereo is the seed growing class of algorithms [5], [6], [4]. The correspondences are grown from a small set of initial correspondence seeds.…”