Merging of feature tracks for camera motion estimation from video

Abstract

In this paper different application scenarios are presented for which the merging of unconnected feature point tracks is essential for successful camera motion estimation and 3D reconstruction from video. The first application is drift removal for sequential camera motion estimation of long sequences. The state-of-the-art in drift removal is to apply a RANSAC approach to find unconnected feature point tracks. In this paper an alternative spectral algorithm for pairwise matching of unconnected feature point tracks is used. It is then shown that the algorithms can be combined and applied to novel scenarios where independent camera motion estimations must be registered into a common global coordinate system. In the first scenario multiple moving cameras, which capture the same scene simultaneously, are registered. A second new scenario occurs in situations where the tracking of feature points during sequential camera motion estimation fails completely, e.g., due to large occluding objects in the foreground, and the unconnected tracks of the independent reconstructions must be merged. Three experiments with challenging real video sequences demonstrate that the presented techniques work in practice.

Bivariate Feature Localization for SIFT Assuming a Gaussian Feature Shape

“…Then, the outliers are removed using the fundamental matrix and the RANSAC algorithm [23]. Finally, the cameras A k and 3D object points P j are estimated and refined using incremental bundle adjustment [1,24]. The incremental bundle adjustment minimizes the reprojection error, which is defined by the distances between an estimated 3D object point P j projected by the estimated camera matrix A k and the detected positions of corresponding 2D feature point p j,k in the images k (see e.g.…”

“…The incremental bundle adjustment minimizes the reprojection error, which is defined by the distances between an estimated 3D object point P j projected by the estimated camera matrix A k and the detected positions of corresponding 2D feature point p j,k in the images k (see e.g. [24]). …”

Inertial Navigation and Mapping for Autonomous Vehicles

“…This way, matches are found using the appearance of feature descriptors and not their locations. Thus, the set of matches which corresponds to the same feature is called a feature track (Thormählen et al, 2008). Such methods are particularly useful if there is a great deal of uncertainty about motion and/or structure or at loop-closing.…”

Interactive Full-Body Motion Capture Using Infrared Sensor Network

“…The theory of SfM reconstruction in this case is: given consistent structure in each image I n , the point P x,y in each image can be calculated by the projection matrix P n . The method used in this paper is similar to the pairwise matching described in [45], we further enhance this technique by constructing matching planes between each coordinate space.…”