Fusing the Old with the New: Learning Relative Camera Pose with Geometry-Guided Uncertainty

Zhuang, Bingbing; Chandraker, Manmohan

doi:10.1109/cvpr46437.2021.00010

Cited by 10 publications

(4 citation statements)

References 70 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The second stage of explicit 2-D map-based relocalization aims to obtain more precise poses of the queries by performing additional relative pose estimation with respect to the retrieved images. Traditionally, this is tackled by epipolar geometry, relying on the 2-D-2-D correspondences determined by local descriptors [93], [94], [95]. In contrast, deep-learning-based approaches regress the relative poses directly from pairwise images.…”

Section: A Relocalization In a 2-d Mapmentioning

confidence: 99%

Deep Learning for Visual Localization and Mapping: A Survey

Chen,

Wang,

et al. 2024

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

Deep-learning-based localization and mapping approaches have recently emerged as a new research direction and receive significant attention from both industry and academia. Instead of creating hand-designed algorithms based on physical models or geometric theories, deep learning solutions provide an alternative to solve the problem in a data-driven way. Benefiting from the ever-increasing volumes of data and computational power on devices, these learning methods are fast evolving into a new area that shows potential to track self-motion and estimate environmental models accurately and robustly for mobile agents. In this work, we provide a comprehensive survey and propose a taxonomy for the localization and mapping methods using deep learning. This survey aims to discuss two basic questions: whether deep learning is promising for localization and mapping, and how deep learning should be applied to solve this problem. To this end, a series of localization and mapping topics are investigated, from the learning-based visual odometry and global relocalization to mapping, and simultaneous localization and mapping (SLAM). It is our hope that this survey organically weaves together the recent works in this vein from robotics, computer vision, and machine learning communities and serves as a guideline for future researchers to apply deep learning to tackle the problem of visual localization and mapping.

show abstract

Section: A Relocalization In a 2-d Mapmentioning

confidence: 99%

Deep Learning for Visual Localization and Mapping: A Survey

Chen,

Wang,

et al. 2024

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

show abstract

“…The importance of using the geometric knowledge about the scene and enforcing the geometric constraints during learning was pointed out in [20], and has been demonstrated recently also by other pose estimation methods. A camera pose estimation pipeline that is applicable for fusing classical geometry and deep learning is proposed in [21], while [22] introduces an end-to-end system consisting of deep learning modules for feature extraction, matching, and outlier rejection while optimising for the camera pose objective.…”

Section: B Camera Pose Estimationmentioning

confidence: 99%

“…Uncertainty in learned feature extractors and camera pose estimation methods seems to be not yet fully exploited, despite its importance in the conventional, geometric approaches. Among the examples considering uncertainty, [27] uses Bayesian neural networks to obtain localisation uncertainty, and a recent approach [21] learns the deep neural network uncertainty guided by the geometric uncertainty. In our localisation scenario, we are interested in aleatoric uncertainty, which depends on the inputs, and may be estimated from data [28].…”

Section: Detection Of Keypointsmentioning

confidence: 99%

Geometry-Aware Keypoint Network: Accurate Prediction of Point Features in Challenging Scenario

Nowak¹,

Skrzypczyński²

2022

Annals of Computer Science and Information Systems

View full text Add to dashboard Cite

In this paper, we consider a challenging scenario of localising a camera with respect to a charging station for electric buses. In this application, we face a number of problems, including a substantial scale change as the bus approaches the station, and the need to detect keypoints on a weakly textured object in a wide range of lighting and weather conditions. Therefore, we use a deep convolutional neural network to detect the features, while retaining a conventional procedure for pose estimation with 2D-to-3D associations. We leverage here the backbone of HRNet, a state-of-the-art network used for detection of feature points in human pose recognition, and we further improve the solution adding constraints that stem from the known scene geometry. We incorporate the reprojection-based geometric priors in a novel loss function for HRNet training and use the object geometry to construct sanity checks in postprocessing. Moreover, we demonstrate that our Geometry-Aware Keypoint Network yields feasible estimates of the geometric uncertainty of point features. The proposed architecture and solutions are tested on a large dataset of images and trajectories collected with a real city bus and charging station under varying environmental conditions.

show abstract

“…Secondly, uncertainty is treated as guidance for pseudo label quality estimation for weakly/semisupervised learning [55,80]. Thirdly, uncertainty serves as extra information in addition to the task related prediction, leading to error-awareness models for fully-supervised learning [15,44,48,64,68,69,84]. Although many uncertainty related applications [5,7,10,13,42,46,59,65,76,78] have been proposed, we notice they focus on directly using uncertainty without thoroughly analysing the limitations of the uncertainty estimation techniques.…”

Section: Uncertainty Related Applicationsmentioning

confidence: 99%

Dense Uncertainty Estimation via an Ensemble-based Conditional Latent Variable Model

Zhang¹,

Dai²,

Harandi³

et al. 2021

Preprint

View full text Add to dashboard Cite

Uncertainty estimation has been extensively studied in recent literature, which can usually be classified as aleatoric uncertainty and epistemic uncertainty. In current aleatoric uncertainty estimation frameworks, it is often neglected that the aleatoric uncertainty is an inherent attribute of the data and can only be correctly estimated with an unbiased oracle model. Since the oracle model is inaccessible in most cases, we propose a new sampling and selection strategy at train time to approximate the oracle model for aleatoric uncertainty estimation. Further, we show a trivial solution in the dual-head based heteroscedastic aleatoric uncertainty estimation framework and introduce a new uncertainty consistency loss to avoid it. For epistemic uncertainty estimation, we argue that the internal variable in a conditional latent variable model is another source of epistemic uncertainty to model the predictive distribution and explore the limited knowledge about the hidden true model. We validate our observation on a dense prediction task, i.e., camouflaged object detection. Our results show that our solution achieves both accurate deterministic results and reliable uncertainty estimation.

show abstract

Fusing the Old with the New: Learning Relative Camera Pose with Geometry-Guided Uncertainty

Cited by 10 publications

References 70 publications

Deep Learning for Visual Localization and Mapping: A Survey

Deep Learning for Visual Localization and Mapping: A Survey

Geometry-Aware Keypoint Network: Accurate Prediction of Point Features in Challenging Scenario

Dense Uncertainty Estimation via an Ensemble-based Conditional Latent Variable Model

Contact Info

Product

Resources

About