Concurrent 3-D motion segmentation and 3-D interpretation of temporal sequences of monocular images

Abstract

The purpose of this study is to investigate a variational method for joint multiregion three-dimensional (3-D) motion segmentation and 3-D interpretation of temporal sequences of monocular images. Interpretation consists of dense recovery of 3-D structure and motion from the image sequence spatiotemporal variations due to short-range image motion. The method is direct insomuch as it does not require prior computation of image motion. It allows movement of both viewing system and multiple independently moving objects. The problem is formulated following a variational statement with a functional containing three terms. One term measures the conformity of the interpretation within each region of 3-D motion segmentation to the image sequence spatiotemporal variations. The second term is of regularization of depth. The assumption that environmental objects are rigid accounts automatically for the regularity of 3-D motion within each region of segmentation. The third and last term is for the regularity of segmentation boundaries. Minimization of the functional follows the corresponding Euler-Lagrange equations. This results in iterated concurrent computation of 3-D motion segmentation by curve evolution, depth by gradient descent, and 3-D motion by least squares within each region of segmentation. Curve evolution is implemented via level sets for topology independence and numerical stability. This algorithm and its implementation are verified on synthetic and real image sequences. Viewers presented with anaglyphs of stereoscopic images constructed from the algorithm's output reported a strong perception of depth.

Efficient Dense Rigid-Body Motion Segmentation and Estimation in RGB-D Video

“…Weber and Malik [1997] proposed dense 3D motion segmentation between monocular images from optical flow assuming an affine camera model. Sekkati and Mitiche [2006] tackle dense 3D multibody structure-from-motion (SfM) from monocular video in a variational framework and demonstrate qualitative results. Recently, a variational framework has been proposed that integrates rigid-body motion segmentation with dense 3D reconstruction [Roussos et al, 2012] from monocular image sequences.…”

“…Most recent methods for dense 3D motion segmentation are still far from real-time performance [Sekkati and Mitiche, 2006, Zhang et al, 2011, Wang et al, 2012, Roussos et al, 2012.…”

Efficient Dense 3D Rigid-Body Motion Segmentation in RGB-D Video

“…Many approaches match images sparsely at interest points and infer the groups of points with common 3D rigid-body motion [1,9,12,13,15]. Methods for dense 3D motion segmentation are still far from real-time performance [14,16,23,25].…”

Motion segmentation and 3D positioning of multiple mobile robots using an array of static cameras in an intelligent space

“…Robot segmentation and position are obtained through the minimization of an objective function. There are many works which use an objective function [5] [6]. However, these works present several disadvantages such as high computational cost, reliance on the initial values of the variables, or previous knowledge about the number of mobile robots.…”