Efficient Non-consecutive Feature Tracking for Structure-from-Motion

Abstract

Abstract. Structure-from-motion (SfM) is an important computer vision problem and largely relies on the quality of feature tracking. In image sequences, if disjointed tracks caused by objects moving in and out of the view, occasional occlusion, or image noise, are not handled well, the corresponding SfM could be significantly affected. In this paper, we address the non-consecutive feature point tracking problem and propose an effective method to match interrupted tracks. Our framework consists of steps of solving the feature 'dropout' problem when indistinctive structures, noise or even large image distortion exist, and of rapidly recognizing and joining common features located in different subsequences. Experimental results on several challenging and large-scale video sets show that our method notably improves SfM.

Improved 3D sparse maps for high-performance SFM with low-cost omnidirectional robots

“…Central to the SfM problem is the concept of feature matching and tracking [17,13,18]. Generally, prior work concentrates on variation and feature type, rather than the cross-image tracking methodology.…”

Foreground Segmentation from Occlusions Using Structure and Motion Recovery

“…In case of video with small displacements between two frames, feature tracking methods like KLT [19] tend to produce less outliers and provide increased localization accuracy compared to feature matching methods [20]. Methods as presented in [6,7,8] additionally make use of feature correspondences in non-consecutive frames as shown in Fig. 2 and therefore increase the reconstruction reliability.…”

“…A reappearing feature induces a new 3D object point which adopts a different and therefore erroneous position. Recent approaches solve this problem by incorporating non-consecutive feature correspondences [5,6,7,8]. The additional Fig.…”

“…The resulting trajectories are incorporated in an extended bundle adjustment optimization for the camera estimation. The additional constraints lead to an improved scene reconstruction [7,8]. We identify foreground objects in the camera images as regions which occlude already reconstructed scene content.…”

Occlusion Handling for the Integration of Virtual Objects Into Video

“…A reappearing feature induces a new 3D object point which adopts a different and therefore erroneous position. Recent approaches solve this problem by incorporating non-consecutive feature correspondences Engels et al, 2008;Zhang et al, 2010;Cordes et al, 2011). The additional correspondences and their trajectories are used to stabilize the bundle adjustment and improve the reconstruction results.…”