Investigating the Role of Image Retrieval for Visual Localization

Humenberger, Martin; Cabon, Yohann; Pion, Noé; Weinzaepfel, Philippe; Lee, Donghwan; Guérin, Nicolas; Sattler, Torsten

doi:10.1007/s11263-022-01615-7

Cited by 20 publications

(5 citation statements)

References 117 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Relative camera pose regression [161], [162], [163], [164] techniques produce reference images. For the relative camera pose, regression may be computed using the previous image retrieval method [165], which first determines the image in the database that is most similar to the query image, and then calculates the absolute pose of the target image after predicting their respective relative poses. NNnet [59] initially proposed an image-retrieval-based relative pose regression approach.…”

Section: ) Relative Pose Regressionmentioning

confidence: 99%

A Review of Recurrent Neural Network Based Camera Localization for Indoor Environments

et al. 2023

View full text Add to dashboard Cite

Camera localization involves the estimation of the camera pose of an image from a random scene. We used a single image or sequence of images or videos as the input. The output depends on the representation of the scene and method used. Several computer vision applications, such as robot navigation and safety inspection, can benefit from camera localization. Camera localization is used to determine the position of an object on the camera in an image containing multiple images in a sequence. Structure-based localization techniques have achieved considerable success owing to a combination of image matching and coordinate regression. Absolute and relative pose regression techniques can provide end-to-end learning; however, they exhibit poor accuracies. Despite the rapid growth in computer vision, there has been no thorough review of the categorization, evaluation, and synthesis of structures and regression-based techniques. Input format and loss strategies for recurrent neural networks (RNN) have not been adequately described in the literature. The main topic is indoor camera pose regression, which is a part of the camera localization techniques. First, we discuss certain application areas for camera localization. We then discuss different camera localization techniques, such as feature and structure-based, absolute and relative pose regression techniques, and simultaneous localization and mapping (SLAM). We evaluated the frequently used datasets and qualitatively compared the absolute and relative camera pose estimation approaches. Finally, we discuss potential directions for future research, such as optimizing the computational cost of the features and evaluating the end-to-end characteristics of multiple cameras.INDEX TERMS Camera pose regression, absolute pose regression, indoor positioning, camera localization, robot navigation, SLAM.

show abstract

Section: ) Relative Pose Regressionmentioning

confidence: 99%

A Review of Recurrent Neural Network Based Camera Localization for Indoor Environments

et al. 2023

View full text Add to dashboard Cite

show abstract

“…(1) with image retrieval (IR), (2) by establishing correspondences between 2D pixels and 3D scene coordinates and applying PnP-RANSAC (assuming known camera intrinsics) (3) by regressing explicit camera parameters from which the camera pose can be recovered (e.g., the camera pose parameters or a relative camera pose). IR methods [43,1,15,31,19] can localize images by taking the pose of the closest neighbor as the pose of the query image, or by interpolating the poses of several neighbors [21,35]. Methods in the second class, often referred to as 'structure-based', include SLPs and SCR approaches.…”

Section: Visual Localizationmentioning

confidence: 99%

Learning to Localize in Unseen Scenes with Relative Pose Regressors

Idan¹,

Shavit²,

Keller³

2023

Preprint

View full text Add to dashboard Cite

Relative pose regressors (RPRs) localize a camera by estimating its relative translation and rotation to a poselabelled reference. Unlike scene coordinate regression and absolute pose regression methods, which learn absolute scene parameters, RPRs can (theoretically) localize in unseen environments, since they only learn the residual pose between camera pairs. In practice, however, the performance of RPRs is significantly degraded in unseen scenes. In this work, we propose to aggregate paired feature maps into latent codes, instead of operating on global image descriptors, in order to improve the generalization of RPRs. We implement aggregation with concatenation, projection, and attention operations (Transformer Encoders) and learn to regress the relative pose parameters from the resulting latent codes. We further make use of a recently proposed continuous representation of rotation matrices, which alleviates the limitations of the commonly used quaternions. Compared to stateof-the-art RPRs, our model is shown to localize significantly better in unseen environments, across both indoor and outdoor benchmarks, while maintaining competitive performance in seen scenes. We validate our findings and architecture design through multiple ablations. Our code and pretrained models is publicly available 1 .

show abstract

“…Recently, many studies have focused on visual localization, either in indoor or outdoor environments, with almost the same technical route [27][28][29]. The users' positions are always estimated by query images in the condition of known or unknown camera-intrinsic parameters.…”

Section: Related Workmentioning

confidence: 99%

Hierarchical Clustering-Based Image Retrieval for Indoor Visual Localization

et al. 2022

View full text Add to dashboard Cite

Visual localization is employed for indoor navigation and embedded in various applications, such as augmented reality and mixed reality. Image retrieval and geometrical measurement are the primary steps in visual localization, and the key to improving localization efficiency is to reduce the time consumption of the image retrieval. Therefore, a hierarchical clustering-based image-retrieval method is proposed to hierarchically organize an off-line image database, resulting in control of the time consumption of image retrieval within a reasonable range. The image database is hierarchically organized by two stages: scene-level clustering and sub-scene-level clustering. In scene-level clustering, an improved cumulative sum algorithm is proposed to detect change points and then group images by global features. On the basis of scene-level clustering, a feature tracking-based method is introduced to further group images into sub-scene-level clusters. An image retrieval algorithm with a backtracking mechanism is designed and applied for visual localization. In addition, a weighted KNN-based visual localization method is presented, and the estimated query position is solved by the Armijo–Goldstein algorithm. Experimental results indicate that the running time of image retrieval does not linearly increase with the size of image databases, which is beneficial to improving localization efficiency.

show abstract

Investigating the Role of Image Retrieval for Visual Localization

Cited by 20 publications

References 117 publications

A Review of Recurrent Neural Network Based Camera Localization for Indoor Environments

A Review of Recurrent Neural Network Based Camera Localization for Indoor Environments

Learning to Localize in Unseen Scenes with Relative Pose Regressors

Hierarchical Clustering-Based Image Retrieval for Indoor Visual Localization

Contact Info

Product

Resources

About