Drift-Free Real-Time Sequential Mosaicing

Abstract

We present a sequential mosaicing algorithm for a calibrated rotating camera which can for the first time build drift-free, consistent spherical mosaics in real-time, automatically and seamlessly even when previously viewed parts of the scene are re-visited. Our mosaic is composed of elastic triangular tiles attached to a backbone map of feature directions over the unit sphere built using a sequential EKF SLAM (Extend Kalman Filter Simultaneous Localization And Mapping) approach.This method represents a significant advance on previous mosaicing techniques which either require off-line optimization or which work in real-time but use local alignment of nearby images and ultimately drift. We demonstrate the system's real-time performance with real-time mosaicing results from sequences with 360 degrees pan. The system shows good global mosaicing ability despite the challenging conditions: hand-held simple low-resolution webcam, varying natural outdoor illumination, and people moving in the scene.

Exploiting distinctive visual landmark maps in pan–tilt–zoom camera networks

“…Real-time global optimization with on-line SLAM applied over a number of frames has been implemented in several systems [8,25,41] considering cameras with pre-calculated internal calibration. However all these systems only consider the case of camera panning and tilting with smooth motion.…”

Device-tagged feature-based localization and mapping of wide areas with a PTZ camera

“…On the contrary PTZ-SLAM must solve for 8DOF (three for the pose and five for the intrinsic parameters) and deals with dynamic environments. Civera et al [6] present a sequential mosaicing algorithm for an internal calibrated rotating camera using an Extended Kalman Filter SLAM approach. Authors point out that their results are mainly occurred because a rotating camera is a constrained and well-understood linear problem.…”

Real-Time Spherical Mosaicing Using Whole Image Alignment

“…This methodology was recently successfully applied to image mosaicing by Civera et al [6], in the first work which was able to demonstrate drift-free mosaicing at frame-rate from a rotating camera. The computational cost of the EKF backbone of this technique, however, scales badly with the number of features kept in the map state, and this meant that only around 10-15 features (matched using 11×11 pixel patches) could be tracked per frame; all but 3% of every image was ignored for the purposes of image alignment, and this sets a limit on the mosaicing quality which can be achieved.…”