A Variational Framework for Structure from Motion in Omnidirectional Image Sequences

Abstract

We address the problem of depth and egomotion estimation from omnidirectional images. We propose a correspondence-free structure-from-motion problem for sequences of images mapped on the 2-sphere. A novel graph-based variational framework is first proposed for depth estimation between pairs of images. The estimation is cast as a TV-L1 optimization problem that is solved by a fast graph-based algorithm. The ego-motion is then estimated directly from the depth information without explicit computation of the optical flow. Both problems are finally addressed together in an iterative algorithm that alternates between depth and ego-motion estimation for fast computation of 3D information from motion in image sequences. Experimental results demonstrate the effective performance of the proposed algorithm for 3D reconstruction from synthetic and natural omnidirectional images.

Foreground silhouette extraction robust to sudden changes of background appearance

“…Figure 4 presents the silhouettes extracted given such approach. It can be seen that the (a) Original image (b) Graph Cuts [12] (c) TV framework [13] (d) Our approach Fig. 4: Foreground silhouettes obtained by temporal depth variation using two depth estimation techniques.…”

“…Since it is an off-line process, there is no constraint on the computational complexity. We use the TV 1 framework proposed in [13] to compute the disparity background.…”

“…Although a lot of effort has been dedicated to find accurate algorithms to estimate disparity maps, the performances are often poor and not consistent across scenes. Algorithms often perform poorly on uniform and non-textured (a) Frame t 0 (b) Block Matching [11] (c) Graph Cuts [12] (d) TV framework [13] (a) Frame t 1 (b) Block Matching [11] (c) Graph Cuts [12] (d) TV framework [13] regions (See Figure 2). Stereo mismatch algorithms compute the background disparity only.…”

Plenoptic based super-resolution for omnidirectional image sequences

“…We obtain such estimates in a pre-processing step using the same structure from motion algorithm described in [7]. For each couple of successive frames li and lj the camera ego-motion and the depth map Di(ω k ) are jointly estimated in variational framework.…”

“…Since the camera motion and the structure of the scene are unknown, we estimate them as described in [7]. We then use the plenoptic geometry of the scene to perform a registration step between successive frames of the video sequence, and we exploit all the visual information for the generation of a high resolution spherical image.…”