Efficient Dense 3D Rigid-Body Motion Segmentation in RGB-D Video

Abstract

Motion is a fundamental segmentation cue in video. Many current approaches segment 3D motion in monocular or stereo image sequences, mostly relying on sparse interest points or being dense but computationally demanding. We propose an efficient expectation-maximization (EM) framework for dense 3D segmentation of moving rigid parts in RGB-D video. Our approach segments two images into pixel regions that undergo coherent 3D rigid-body motion. Our formulation treats background and foreground objects equally and poses no further assumptions on the motion of the camera or the objects than rigidness. While our EM-formulation is not restricted to a specific image representation, we supplement it with efficient image representation and registration for rapid segmentation of RGB-D video. In experiments we demonstrate that our approach recovers segmentation and 3D motion at good precision.

An integrated approach to visual perception of articulated objects

“…Ren et al present a method to simultaneously track and reconstruct 3-D objects [16] by refining an initial primitive shape model; in contrast to our method it can only reconstruct and track one moving object and the initial location of this object must be manually provided. Stückler et al suggest a method which combines object tracking, segmentation, and reconstruction using an EM algorithm [5]. Their method differs from ours, as it relies on an initial oversegmentation and groups the segments using motion and surface clues, making it sensitive to a wrong initial segmentation.…”

“…Our method extends two recent methods that follow the same insight [5,6]. Most importantly, it includes and exploits the estimation of kinematic structures for interacted articulated objects (Fig.…”

“…We distinguish two main approaches to visual tracking: based on a known shape model of the object [2], and based on sufficient texture using point features [4] or dense optical flow [5,7]. While tracking with a known model is more accurate, texturebased approaches are also applicable to unknown objects.…”

“…To detect which points moved due to object motion, image differencing [13,14] can be applied. To reject changes caused by background motion it can be combined with information from a tracker to select only points that move consistently with the object [5,7]. In our approach we apply motion-based segmentation to generate sparse segments and extend them using supervoxel region growing, and we use single-image segmentation by using the continuously updated reconstruction of the shape.…”

Efficient Dense Rigid-Body Motion Segmentation and Estimation in RGB-D Video

“…In experiments, we demonstrate that our approach efficiently identifies moving segments with high accuracy and recovers 3D rigid-body motion of the segments at good precision. This article extends the work in [Stückler and Behnke, 2013] with a detailed derivation of our method and further comparative evaluation.…”