Wide baseline pose estimation from video with a density-based uncertainty model

Abstract

Robust wide baseline pose estimation is an essential step in the deployment of smart camera networks. In this work, we highlight some current limitations of conventional strategies for relative pose estimation in difficult urban scenes. Then, we propose a solution which relies on an adaptive search of corresponding interest points in synchronized video streams which allows us to converge robustly toward a high-quality solution. The core idea of our algorithm is to build across the image space a nonstationary mapping of the local pose estimation uncertainty, based on the spatial distribution of interest points. Subsequently, the mapping guides the selection of new observations from the video stream in order to prioritize the coverage of areas of high uncertainty. With an additional step in the initial stage, the proposed algorithm may also be used for refining an existing pose estimation based on the video data; this mode allows for performing a data-driven self-calibration task for stereo rigs for which accuracy is critical, such as onboard medical or vehicular systems. We validate our method on three different datasets which cover typical scenarios in pose estimation. The results show a fast and robust convergence of the solution, with a significant improvement, compared to single image-based alternatives, of the RMSE of ground-truth matches, and of the maximum absolute error.

GPU-accelerated Height Map Estimation with Local Geometry Priors in Large Scenes

“…In [11], the authors propose an unsupervised pedestrian detection method based on information derived from mul-1 tiple camera sources. They formulate the detector as a Markov Random Field (MRF) based stereo matcher, which has to minimize a global energy by assigning some labels to each pixel p of the reference image I:…”

“…On the other hand, a low-level fusion strategy, such as the one proposed in [2], is applied before the detection step and is expected to cope better with ambiguous scenes specific to crowded environments. Recently, the work of [11] redefined the low-level fusion as a global height map estimation taking into account the local geometry of the scene, represented by the accurate ground plane location and the vertical vanishing line direction (Figure 1). The optimization was solved using a Loopy Belief Propagation algorithm, but the price for a high-quality solution compared to a heuristic based one is the computational cost (around 18 minutes per triplet of synchronized frames).…”

“…The objective of the current work is to reformulate the algorithm proposed in [11] and to accelerate it by exploiting a massively parallel architecture, in order to render it compatible with practical applications. Section 2 offers a synthetic overview of the initial algorithm, then in Section 3 we highlight at different levels the various optimizations which were performed in order to adapt the algorithm to the considered architecture.…”

Constraining Relative Camera Pose Estimation with Pedestrian Detector-Based Correspondence Filters

“…Secondly, the pedestrians tend to have a homogeneous appearance and are prone to generating outlier observations in the form of wrong correspondences. The solution provided by standard guided matching techniques [18] tends to drift progressively and include outliers, but recently [11,12] proposed a guided matching algorithm aiming to enforce a uniform selection of matches in the common field of view of the cameras. This in turn provides a high-quality estimation locally as well.…”

Online extrinsic calibration of multiple long baseline ranging sensors

“…In [71], Pellicanò et al proposes an online wide baseline pose estimation method for a camera network. It considers the unbalanced distribution of the feature points in the images and builds a moving mapping of the local pose estimation uncertainty, based on the spatial distribution of feature points.…”