Extraction of 3D freeform surfaces as visual landmarks for real-time tracking

Abstract

This work presents a system for the generation of a free-form surface model from video sequences. Although any single centered camera can be applied in the proposed system the approach is demonstrated using fisheye lenses because of their good properties for tracking. The system is designed to function automatically and to be flexible with respect to size and shape of the reconstructed scene. To minimize geometric assumptions a statistic fusion of dense depth maps is utilized. Special attention is payed to the necessary rectification of the spherical images and the resulting iso-disparity surfaces, which can be exploited in the fusion approach. Before dense depth estimation can be performed the cameras' pose parameters are extracted by means of a structure-from-motion (SfM) scheme. In this respect automation of the system is achieved by thorough decision model based on robust statistics and error propagation of projective measurement uncertainties. This leads to a scene-independent set of only a few parameters. All system components are formulated in a general way, making it possible to cope with any single centered projection model, in particular with spherical cameras. In using wide field-of-view cameras the presented system is able to reliably retrieve poses and consistently reconstruct large scenes. A textured triangle mesh constructed on basis of the scene's reconstructed depth, makes the system's results suitable to function as reference models in a GPU driven analysis-by-synthesis framework for real-time tracking.

ToF-sensors: New dimensions for realism and interactivity

“…Figure 2 shows two sample scenes acquired with this kind of approach. For high quality reconstruction, the low resolution and small field of view of a ToF-camera can be compensated for by combining it with high-resolution imagebased 3D scene reconstruction, for example by utilizing a structure-from-motion (SFM) approach [1,16]. The inherent problem of SFM, that no metric scale can be obtained, is solved by the metric properties of the ToF-measurements.…”

Robust GPU-assisted camera tracking using free-form surface models

“…The textured triangle mesh generated by the offline phase [4] is a reconstructive model of the scene: it represents the scene well in the sense that we can reconstruct a virtual view of the scene from any given viewpoint. However, when the camera is switched on during the online phase, no prior information is given about the viewpoint and we have to initially register the camera with respect to the scene.…”

“…Our system overcomes this tradeoff in that we bring forward the bundle adjustment in an offline phase [4], where we do not have time constraints. The results are stored and can be used in an efficient way in the online phase.…”

“…fish-eye with 180°) is systematically moved within a scene and a structure-from-motion-approach is applied to reconstruct the environment as a textured triangle mesh. Since no time-constraints are imposed during offline-processing, an optimal batch tracking with bundle adjustment and multi-camera depth estimation is possible, yielding high quality models [4]. Next, robust 2D features (e.g.…”

FPGA implementation of an efficient similarity-based adaptive window algorithm for real-time stereo matching

“…The perception of the depth values of the points contained in a scene is one of the most important tasks of the computer vision systems and has been used in several applications such as positioning systems for mobile robots and both recognition, detection and three-dimensional reconstruction of objects [2,3,4,13,24].…”