Time-sequential extraction of motion layers

Abstract

A new time-sequential approach for motion layer extraction is presented. We assume that the scene can be described by a set of layers associated to affine motion models. In one or more key frames, the segmentation is obtained using a semi-automatic method. At a subsequent instant, the first step of the proposed algorithm is the prediction of the segmentation from one image to the next one, using motion models estimated for each layer. The second step is the refinement of the predicted motion boundaries by graph cut. Only the appearing areas and a strip around the predicted boundaries are questioned. A new rigidity constraint improves the temporal consistency of foreground rigid objects. Experimental results show that our sequential approach is at least as effective as more complex simultaneous approaches while being less computationally demanding.

EXCOL: An EXtract-and-COmplete Layering Approach to Cartoon Animation Reusing

“…In Figure 13, we compare the performance of EXCOL with layer extraction methods of bi-layer segmentation [13], time-sequential graph restriction (TSGR) [6] and multi-label propagation (MLP) approach [15]. Clearly EX-COL achieves the best layer extraction.…”

“…The right example is achieved by applying the approach of [13] iteratively. 4th Row: Results with TSGR [6]. 5st Row: Results with MLP [15].…”

“…Like most of interactive video segmentation techniques (e.g., [6], [18]), we let the user provide guidance on some frame, which we call a key frame. The key frame, denoted as k 0 , is typically the frame where desired foreground layers have little overlapping, which ease the layer completion step, as we will show later.…”

“…Vision-based layering techniques typically assume that a scene can be approximately decomposed into several layers, each of which is consistently governed by a 2D motion model. Existing methods mainly vary with motion models (e.g., dense motion fields represented by optical flow [11], parametric motion models represented as affine transformations [6], [12], homography [10]) and motion segmentation algorithms for estimating motion model parameters (e.g., based on Kmeans [11], linear space [12], graph cut [6], [10], [13]). However, due to motion approximation errors, vision-based layering techniques often suffer from artifacts especially at regions near to layer boundaries.…”

“…Those vision-based techniques are commonly designed for general types of videos and typically use parametric motion models (e.g., based on an affine motion model [6]) for describing scene content within individual layers. However such motion models are generally not expressive enough for capturing highly exaggerated motions often exhibited in cartoon animations [1], making such vision-based techniques not suitable for cartoon animation layering.…”