Drift-free pose estimation with hemispherical cameras

The term ''Augmented Reality'' covers a wide range of applications, from the overlay of virtual graphics on a real scene using a head-mounted display for use in areas such as industrial maintenance, through to the insertion of real-time virtual graphics in TV programmes. Fundamental to all these applications is the need to be able to accurately track the motion of the camera, so that the graphics may be rendered so as to appear rigidly locked to the real world. To overcome the limitations of existing tracking systems, the MATRIS project has developed a real-time system for measuring the movement of a camera, which uses image analysis to track naturally occurring features in the scene, and data from an inertial sensor. No additional sensors, special markers, or camera mounts are required. This paper gives an overview of the system, provides the context for the other articles in this journal and presents some results.

Section: Augmented Realitymentioning

confidence: 99%

Section: Augmented Realitymentioning

confidence: 99%

Section: Fisheye Cameramentioning

confidence: 99%

See 1 more Smart Citation

The MATRIS project: real-time markerless camera tracking for Augmented Reality and broadcast applications

Chandaria¹,

Thomas²,

Stricker

2007

“…In [33] it has been shown that no significant reduction of pose error could be observed when iterating more than two or three times. If the prediction is already good and if performance is an issue, tracking with fewer or even only one rendering iteration is feasible.…”

Section: Looped Renderingmentioning

confidence: 99%

Robust GPU-assisted camera tracking using free-form surface models

Koeser

Bartczak

Koch

2007

We propose a marker-less model-based camera tracking approach, which makes use of GPU-assisted analysis-by-synthesis methods on a very wide field of view (e.g. fish-eye) camera. After an initial registration based on a learned database of robust features, the synthesis part of the tracking is performed on graphics hardware, which simulates internal and external parameters of the camera, this way minimizing lens and viewpoint differences between a model view and a real camera image. Based on an automatically reconstructed free-form surface model we analyze the sensitivity of the tracking to the model accuracy, in particular the case when we represent curved surfaces by planar patches. We also examine accuracy and show on synthetic and on real data that the system does not suffer from drift accumulation. The wide field of view of the camera and the subdivision of our reference model into many textured free-form surface patches make the system robust against illumination changes, moving persons and other occlusions within the environment and provide a camera pose estimate in a fixed and known coordinate system.

“…On the other hand, the FreeD system described in [61] does not have these problems but needs an expensively calibrated marker setup, which makes its application outside of prepared studio environments difficult. In order to overcome these limitations [30,31] propose a tracking system which relies on a 3D reference model that was extracted from an image sequence itself. It is proposed to split tracking into two phases: an offline phase where the model is generated and an online phase where the camera is tracked in real-time.…”

Section: Introductionmentioning

confidence: 99%

Extraction of 3D freeform surfaces as visual landmarks for real-time tracking

Bartczak

Koeser

Woelk

et al. 2007

This work presents a system for the generation of a free-form surface model from video sequences. Although any single centered camera can be applied in the proposed system the approach is demonstrated using fisheye lenses because of their good properties for tracking. The system is designed to function automatically and to be flexible with respect to size and shape of the reconstructed scene. To minimize geometric assumptions a statistic fusion of dense depth maps is utilized. Special attention is payed to the necessary rectification of the spherical images and the resulting iso-disparity surfaces, which can be exploited in the fusion approach. Before dense depth estimation can be performed the cameras' pose parameters are extracted by means of a structure-from-motion (SfM) scheme. In this respect automation of the system is achieved by thorough decision model based on robust statistics and error propagation of projective measurement uncertainties. This leads to a scene-independent set of only a few parameters. All system components are formulated in a general way, making it possible to cope with any single centered projection model, in particular with spherical cameras. In using wide field-of-view cameras the presented system is able to reliably retrieve poses and consistently reconstruct large scenes. A textured triangle mesh constructed on basis of the scene's reconstructed depth, makes the system's results suitable to function as reference models in a GPU driven analysis-by-synthesis framework for real-time tracking.