Is Geometry Enough for Matching in Visual Localization?

Zhou, Qunjie; Agostinho, Sérgio; Ošep, Aljoša; Leal-Taixé, Laura

doi:10.1007/978-3-031-20080-9_24

Cited by 21 publications

(4 citation statements)

References 70 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There is also a high risk of leaking personal data into the map. To mitigate this, some works attempt to compress the maps [12,13,39] or use privacy-preserving representations for the scene appearance [20,44,81] or geometry [68,69]. These however either degrade the accuracy significantly or are easily reverted [50].…”

Section: Related Workmentioning

confidence: 99%

SuperGlue: Learning Feature Matching With Graph Neural Networks

Sarlin

DeTone

Malisiewicz

et al. 2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

1,793

1,061

View full text Add to dashboard Cite

Humans can orient themselves in their 3D environments using simple 2D maps. Differently, algorithms for visual localization mostly rely on complex 3D point clouds that are expensive to build, store, and maintain over time. We bridge this gap by introducing OrienterNet, the first deep neural network that can localize an image with sub-meter accuracy using the same 2D semantic maps that humans use. Orienter-Net estimates the location and orientation of a query image by matching a neural Bird's-Eye View with open and globally available maps from OpenStreetMap, enabling anyone to localize anywhere such maps are available. OrienterNet is supervised only by camera poses but learns to perform semantic matching with a wide range of map elements in an end-to-end manner. To enable this, we introduce a large crowd-sourced dataset of images captured across 12 cities from the diverse viewpoints of cars, bikes, and pedestrians. OrienterNet generalizes to new datasets and pushes the state of the art in both robotics and AR scenarios. The code and trained model will be released publicly.

show abstract

Section: Related Workmentioning

confidence: 99%

SuperGlue: Learning Feature Matching With Graph Neural Networks

Sarlin

DeTone

Malisiewicz

et al. 2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

1,793

1,061

View full text Add to dashboard Cite

show abstract

“…Work on privacy-preserving queries to a localization server thus mostly aims at developing features that prevent image recovery [14,26] or on obfuscating the feature geometry [16,42]. Similarly, work on privacy-preserving scene representation aims to obfuscate the geometry [37,41] (although scene geometry can be recovered under certain conditions [7]), splitting the maps over multiple server for increased data security [15], using implicit representations [11], or storing raw geometry without any feature descriptors [52].…”

Section: Related Workmentioning

confidence: 99%

“…Naturally, sending user data to a server, e.g., in the form of images to be localized or 3D maps recorded by users that will be used for localization, raises privacy concerns [9,41,42]. Work on privacy-preserving localization aims to resolve these concerns by ensuring that private details cannot be recovered from the data sent [14,26,42] to or stored on the server [11,11,15,28,36,41,52].…”

Section: Introductionmentioning

confidence: 99%

How Privacy-Preserving are Line Clouds? Recovering Scene Details from 3D Lines

Chelani

Kahl

Sattler

2021

2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

Visual localization is the task of estimating the camera pose from which a given image was taken and is central to several 3D computer vision applications. With the rapid growth in the popularity of AR/VR/MR devices and cloudbased applications, privacy issues are becoming a very important aspect of the localization process. Existing work on privacy-preserving localization aims to defend against an attacker who has access to a cloud-based service. In this paper, we show that an attacker can learn about details of a scene without any access by simply querying a localization service. The attack is based on the observation that modern visual localization algorithms are robust to variations in appearance and geometry. While this is in general a desired property, it also leads to algorithms localizing objects that are similar enough to those present in a scene. An attacker can thus query a server with a large enough set of images of objects, e.g., obtained from the Internet, and some of them will be localized. The attacker can thus learn about object placements from the camera poses returned by the service (which is the minimal information returned by such a service). In this paper, we develop a proof-of-concept version of this attack and demonstrate its practical feasibility. The attack does not place any requirements on the localization algorithm used, and thus also applies to privacy-preserving representations. Current work on privacy-preserving representations alone is thus insufficient.

show abstract

“…Speciale et al [35] explored new geometric scene and query representations [35,36] and proposed pose estimation techniques for those representations. GoMatch [47] is a storage efficient method for geometric matching of 2D keypoints and 3D points that does not require local descriptors. SegLoc [21] achieves storage efficiency by leveraging semantic segmentation-based map and query representations.…”

Section: Related Workmentioning

confidence: 99%

Learning to Detect Scene Landmarks for Camera Localization

Mikšík

DeGol

et al. 2022

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

Camera localization methods based on retrieval, local feature matching, and 3D structure-based pose estimation are accurate but require high storage, are slow, and are not privacy-preserving. A method based on scene landmark detection (SLD) was recently proposed to address these limitations. It involves training a convolutional neural network (CNN) to detect a few predetermined, salient, scene-specific 3D points or landmarks and computing camera pose from the associated 2D-3D correspondences. Although SLD outperformed existing learning-based approaches, it was notably less accurate than 3D structure-based methods. In this paper, we show that the accuracy gap was due to insufficient model capacity and noisy labels during training. To mitigate the capacity issue, we propose to split the landmarks into subgroups and train a separate network for each subgroup. To generate better training labels, we propose using dense reconstructions to estimate visibility of scene landmarks. Finally, we present a compact architecture to improve memory efficiency. Accuracy wise, our approach is on par with state of the art structurebased methods on the INDOOR-6 dataset but runs significantly faster and uses less storage. Code and models can be found at https://github.com/microsoft/ SceneLandmarkLocalization.

show abstract

Is Geometry Enough for Matching in Visual Localization?

Cited by 21 publications

References 70 publications

SuperGlue: Learning Feature Matching With Graph Neural Networks

SuperGlue: Learning Feature Matching With Graph Neural Networks

How Privacy-Preserving are Line Clouds? Recovering Scene Details from 3D Lines

Learning to Detect Scene Landmarks for Camera Localization

Contact Info

Product

Resources

About