2D-3D camera fusion for visual odometry in outdoor environments

Abstract

Abstract-Accurate estimation of camera motion is very important for many robotics applications involving SfM and visual SLAM. Such accuracy is attempted by refining the estimated motion through nonlinear optimization. As many modern robots are equipped with both 2D and 3D cameras, it is both highly desirable and challenging to exploit data acquired from both modalities to achieve a better localization. Existing refinement methods, such as Bundle adjustment and loop closing, may be employed only when precise 2D-to-3D correspondences across frames are available. In this paper, we propose a framework for robot localization that benefits from both 2D and 3D information without requiring such accurate correspondences to be established. This is carried out through a 2D-3D based initial motion estimation followed by a constrained nonlinear optimization for motion refinement. The initial motion estimation finds the best possible 2D-to-3D correspondences and localizes the cameras with respect the 3D scene. The refinement step minimizes the projection errors of 3D points while preserving the existing relationships between images. The problems of occlusion and that of missing scene parts are handled by comparing the image-based reconstruction and 3D sensor measurements. The effect of data inaccuracies is minimized using an M-estimator based technique. Our experiments have demonstrated that the proposed framework allows to obtain a good initial motion estimate and a significant improvement through refinement.

Static-Map and Dynamic Object Reconstruction in Outdoor Scenes Using 3-D Motion Segmentation

“…7a and Fig. 7b show that the static scene part of our map is significantly better than that of [22]. For instance, the red rectangle region in Fig.…”

“…To cover a wide speed range, coarse-to-fine dense Optical Flow [16] tracking algorithm has been adopted. The 3D feature trajectories are then retrieved from 2D feature trajectories after establishing 2D-to-3D correspondences similar to [22]. We define dynamic coverage as the area that the dynamic objects cover in an image.…”

“…6, 7, 8. To illustrate the quality of the reconstructed static-maps, the full scene reconstructions using state-of-the-art method [22] are also shown sidewise. Few 2D frames from the sequence are displayed for the motion visualization.…”

“…Secondly, Fig. 9e shows the multi-frame grouping of the truck's point clouds in a common coordinate frame, obtained using [22]. It goes without saying that this representation can hardly be identified as a truck.…”

Dynamic 3D Scene Reconstruction and Enhancement

“…For the past decades, 3D scene reconstruction has been widely studied due to the need of many applications, such as city map modelling [1], robot navigation [2], autonomous driving [3], etc. Among numerous works in this context, the most representative approaches are: structure-from-motion of image sequence [4], RGB-D data fusion [5], and laser scans registration [6].…”