Robust Motion Segmentation From Pairwise Matches

Abstract

In this paper we address a classification problem that has not been considered before, namely motion segmentation given pairwise matches only. Our contribution to this unexplored task is a novel formulation of motion segmentation as a two-step process. First, motion segmentation is performed on image pairs independently. Secondly, we combine independent pairwise segmentation results in a robust way into the final globally consistent segmentation. Our approach is inspired by the success of averaging methods. We demonstrate in simulated as well as in real experiments that our method is very effective in reducing the errors in the pairwise motion segmentation and can cope with large number of mismatches.

On the Usage of the Trifocal Tensor in Motion Segmentation

“…In [3] it is observed that all the two-frame segmentations involving a fixed image provide -up to a permutation of the motions -a possible solution for clustering points in that image. In order to resolve such ambiguity, permutation synchronization [31] is performed.…”

“…They belong to the second category (i.e., the case of two-frame correspondences represented in the middle of Fig. 1) and are inspired by [3]. The first approach -named TriSeg-addresses motion segmentation in two steps:…”

“…Step 1, we exploit RPA [27] in order to fit multiple trifocal tensors to correspondences. Concerning Step 2, we adapt the method proposed in [3] which was developed for merging results coming from pairs of images -in order to deal with triplets of images. The relevance of the trifocal tensor for addressing motion segmentation in three images was already observed in some early works [41,40,10].…”

“…The second category considers the problem of clustering image points (e.g., SIFT keypoints [26]) into different motions, assuming that matches between pairs of images (i.e., two-frame correspondences) are available only. This task is poorly studied and there are only a few works addressing it [3,2]. The typical scenario involves unstructured/unordered image sets where there are large motions between different frames (e.g., the indoor scenes used in [3,2]).…”

“…This task is poorly studied and there are only a few works addressing it [3,2]. The typical scenario involves unstructured/unordered image sets where there are large motions between different frames (e.g., the indoor scenes used in [3,2]). This finds application in multi-body structure from motion [36], where the objective is to reconstruct a 3D scene containing multiple moving objects.…”