Interactive Motion Segmentation

Abstract

Abstract. Interactive motion segmentation is an important task for scene understanding and analysis. Despite recent progress state-of-theart approaches still have difficulties in adapting to the diversity of spatially varying motion fields. Due to strong, spatial variations of the motion field, objects are often divided into several parts. At the same time, different objects exhibiting similar motion fields often cannot be distinguished correctly. In this paper, we propose to use spatially varying affine motion model parameter distributions combined with minimal guidance via user drawn scribbles. Hence, adaptation to motion pattern variations and capturing subtle differences between similar regions is feasible. The idea is embedded in a variational minimization problem, which is solved by means of recently proposed convex relaxation techniques. For two regions (i.e. object and background) we obtain globally optimal results for this formulation. For more than two regions the results deviate within very small bounds of about 2 to 4 % from the optimal solution in our experiments. To demonstrate the benefit of using both model parameters and spatially variant distributions, we show results for challenging synthetic and real-world motion fields.

Spatially Varying Color Distributions for Interactive Multilabel Segmentation

“…Bai et al [2] estimate foreground and background color likelihoods whose gradients are used to define weighted geodesic distances between each pixel and the user scribbles. Spatial information has also successfully been introduced into kernel density estimators, e.g., in the meanshift approach [10], in segmentation based on geodesic distances [2], or in motion analysis [24]. In [40], [30], [14], [17] distance-based sampling was applied to image matting.…”

Segmentation of Moving Objects by Long Term Video Analysis

“…Several techniques based on graph cuts [11], random walks [32], and intermediate methods [60] have been proposed. The latest techniques are built upon variational convex relaxation methods [66], [19], [38], [45], which avoid the typical discretization artifacts of graph based formulations. The variational technique we propose here is in line with these methods.…”

Space-Varying Color Distributions for Interactive Multiregion Segmentation: Discrete versus Continuous Approaches

“…If we use the spatially constant model the results are comparable to those obtained by Santner et al (RGB color information without texture). They obtain the best results combining CIELab and LBP features in a 21 dimensional vector based on a scribble brush of radius 13. We obtain better results on the benchmark by allowing for spatially varying color models.…”

“…Furthermore, since the relaxed problems (13) and (18) are convex, we will always reach their global minimum independent of the label order. However, ambiguities also occur for PDE-based approaches in the multilabel case.…”

“…This is not reflected by previous data fidelity terms. In [13] we segmented optical flow fields by means of spatially dependent flow vector distributions. The first contribution of this paper is to show how non-iid assumptions can be handled in color density estimation leading to strongly improved results.…”