Geodesic Matting: A Framework for Fast Interactive Image and Video Segmentation and Matting

Abstract

An interactive framework for soft segmentation and matting of natural images and videos is presented in this paper. The proposed technique is based on the optimal, linear time, computation of weighted geodesic distances to user-provided scribbles, from which the whole data is automatically segmented. The weights are based on spatial and/or temporal gradients, considering the statistics of the pixels scribbled by the user, without explicit optical flow or any advanced and often computationally expensive feature detectors. These could be naturally added to the proposed framework as well if desired, in the form of weights in the geodesic distances. An automatic localized refinement step follows this fast segmentation in order to further improve the results and accurately compute the corresponding matte function. Additional constraints into the distance definition permit to efficiently handle occlusions such as people or objects crossing each other in a video sequence. The presentation of the framework is complemented with numerous and diverse examples, including extraction of moving foreground from dynamic background in video, natural and 3D medical images, and comparisons with the recent literature.

Dynamic Color Flow: A Motion-Adaptive Color Model for Object Segmentation in Video

“…The basic idea is to create a variable bandwidth contour adaptive to the local statistics of the DCF model. This is in spirit similar to the variable bandwidth trimap proposed in [4] for the purpose of image matting.…”

“…Global color models. Some modern interactive video cutout systems use global color models, such as the popular choice of global Gaussian mixtures (GMM), to represent the appearance of the dynamic objects, e.g., [2][3][4]. Global color models do not consider the spatial arrangements of color components, thus are robust to object motion.…”

“…Similar to the foreground modeling, the DCF model is applied to the reconstructed clean plate, except that σ i is fixed for every background pixel. Typically for static background σ i is set to [2,4] to compensate for small alignment errors. Progressive Background Completion.…”

“…Appearance complexity refers to the fact that the targeted object could contain very similar, or even the same colors and features as the background, thus distinguishing the object from its background using color information becomes a hard problem. 4 In addition, video objects or backgrounds often exhibit nonuniform motions. Thus, applying to the next frame an appearance model constructed from the current one, will be problematic without correctly adapting it to the new position of the possibly deforming object/background caused by the motion.…”

Interactive Motion Segmentation

“…Independently, interactivity has proven itself as a feasible method to facilitate difficult image segmentation tasks. For instance, Bai and Sapiro [5] presented an interactive framework for image and video segmentation. Their technique calculates weighted geodesic distances to scribbles interactively provided by the user, where the weighting is based on spatial or temporal gradients.…”