Multiple motion segmentation with level sets

Abstract

Segmentation of motion in an image sequence is one of the most challenging problems in image processing, while at the same time one that finds numerous applications. To date, a wealth of approaches to motion segmentation have been proposed. Many of them suffer from the local nature of the models used. Global models, such as those based on Markov random fields, perform, in general, better. In this paper, we propose a new approach to motion segmentation that is based on a global model. The novelty of the approach is twofold. First, inspired by recent work of other researchers we formulate the problem as that of region competition, but we solve it using the level set methodology. The key features of a level set representation, as compared to active contours, often used in this context, are its ability to handle variations in the topology of the segmentation and its numerical stability. The second novelty of the paper is the formulation in which, unlike in many other motion segmentation algorithms, we do not use intensity boundaries as an accessory; the segmentation is purely based on motion. This permits accurate estimation of motion boundaries of an object even when its intensity boundaries are hardly visible. Since occasionally intensity boundaries may prove beneficial, we extend the formulation to account for the coincidence of motion and intensity boundaries. In addition, we generalize the approach to multiple motions. We discuss possible discretizations of the evolution (PDE) equations and we give details of an initialization scheme so that the results could be duplicated. We show numerous experimental results for various formulations on natural images with either synthetic or natural motion.

2D Motion Description and Contextual Motion Analysis: Issues and New Models

“…However, the spatial segmentation stage is often time consuming, and getting an effective improvement on the final motion segmentation accuracy remains questionable. Using the level-set framework is another way to precisely locate region boundaries while dealing with topology changes [38,39], but handling a competitive motion partioning of the image (with the number of regions a priori unknown) remains an open issue in that context even if recent attempts have been reported [11,26].…”

Energy Feature Integration for Motion Segmentation

“…Most motion segmentation models are based on the estimation of optical flow, i.e., the 2D velocity of image points or regions, based on the variation of their intensity values. Mansouri and Konrad (2003) have employed optical flow estimation to segmentation with an active model. They propose a competition approach based on a level set representation.…”

Segmentation of non-rigid video objects using long term temporal consistency

“…Furthermore energetic model does not well represent what occurs in occlusion areas. Labeling in such areas may then be quite random such as observed in [3].…”

“…Many approaches have already been proposed to realize segmentation, such as motion detection with regularization constraints (e.g. Markov Random Fields) [1], region growing [2], or active contours [3,4]. In [5], it has been shown that most of those approaches can be unified as an energetic modeling problem.…”

“…In [5], it has been shown that most of those approaches can be unified as an energetic modeling problem. Segmentation models used in the context of the famous Level Set approach are especially using this kind of model [3,4].…”