Matching Real and Synthetic Panoramic Images Using a Variant of Geometric Hashing

Abstract

ABSTRACT:This work demonstrates an approach to automatically initialize a visual model-based tracker, and recover from lost tracking, without prior camera pose information. These approaches are commonly referred to as tracking-by-detection. Previous tracking-by-detection techniques used either fiducials (i.e. landmarks or markers) or the object's texture. The main contribution of this work is the development of a tracking-by-detection algorithm that is based solely on natural geometric features. A variant of geometric hashing, a model-to-image registration algorithm, is proposed that searches for a matching panoramic image from a database of synthetic panoramic images captured in a 3D virtual environment. The approach identifies corresponding features between the matched panoramic images. The corresponding features are to be used in a photogrammetric space resection to estimate the camera pose. The experiments apply this algorithm to initialize a model-based tracker in an indoor environment using the 3D CAD model of the building. INTRODUCTION TO MODEL-BASED TRACKINGModel-based visual tracking is typically used to obtain the position and the orientation (pose) of the camera when a complete or partial model of the environment pre-exists. (1985) provide comprehensive surveys of model-to-image matching and registration. A typical workflow of model-based tracking is: 1) In the initialization phase, the initial camera pose is estimated without knowledge of the previous pose. Corresponding features (e.g. points or lines) between a camera's image and the 3D model are found either manually or automatically. The pose is resolved from the correspondences, for example using photogrammetric space resection via the collinearity equations. 2) In the frame-to-frame tracking phase, the camera pose is constantly updated by maintaining the model-to-image correspondences when the camera moves. As in the initialization phase, the pose is resolved from the correspondences.Li-Chee-Ming and Armenakis (2016) provide an application of a model-based tracker called ViSP (Visual Servoing Platform) (Comport et al., 2003), for pose estimation in indoor environments. ViSP uses the camera pose from the previous epoch to back-project a 3D model onto the current epoch's image. Edges are matched between the image and the backprojected model. Corresponding edges are used to estimate the camera pose of the current frame. Figure 1 shows experimental results. As ViSP is a sequential algorithm, requiring the camera pose from the previous epoch, a user must manually input the camera pose at the initialization and recovery phases. This work proposed a solution that automats these processes. Initialization and Recovery 1.1Initialization is performed when the tracking starts; the initial pose of the system is estimated without knowledge of the previous camera pose. Pose recovery refers to estimating the pose when tracking is lost. In both cases, the matches provide the 2D to 3D correspondences needed for camera pose estimation. The difficulty in implemen...