Leveraging EfficientNet and Contrastive Learning for Accurate Global-scale Location Estimation

Abstract

In this paper, we address the problem of global-scale image geolocation, proposing a mixed classification-retrieval scheme. Unlike other methods that strictly tackle the problem as a classification or retrieval task, we combine the two practices in a unified solution leveraging the advantages of each approach with two different modules. The first leverages the EfficientNet architecture to assign images to a specific geographic cell in a robust way. The second introduces a new residual architecture that is trained with contrastive learning to map input images to an embedding space that minimizes the pairwise geodesic distance of same-location images. For the final location estimation, the two modules are combined with a search-within-cell scheme, where the locations of most similar images from the predicted geographic cell are aggregated based on a spatial clustering scheme. Our approach demonstrates very competitive performance on four public datasets, achieving new state-of-the-art performance in fine granularity scales, i.e., 15.0% at 1km range on Im2GPS3k. CCS CONCEPTS• Computing methodologies → Computer vision problems;• Information systems → Geographic information systems.

Rethinking Visual Geo-localization for Large-Scale Applications

“…At test time, the trained model is used to extract image descriptors and perform a classic retrieval. While CosPlace might appear similar to previous classificationbased works [25,30,35,42,52], given that they also partition a map into classes, there are substantial differences. These prior works tackle the task of global classification and group images within very large cells (up to hundreds of kilometers wide), building on the idea that nearer scenes have similar semantics (e.g.…”

“…Visual geo-localization as classification. An alternative approach to visual geo-localization is to consider it a classification problem [25,30,35,42,52]. These works build on the idea that two images coming from the same geographical region, although representing different scenes, are likely to share similar semantics, such as architectural styles, types of vehicles, vegetation, etc.…”

“…On the other hand, our partitioning strategy is designed to leverage the availability of dense data and ensure that if two images are from the same class, they visualize the same scene. Moreover, unlike [25,30,35,42,52], once trained our method can be used to perform geo-localization through image retrieval on any given geographical area. 13k Eynsham [13] 48k St Lucia [34] 33k NCLT [7] 3.8M Oxford RobotCar [33] 27k CMU [2] 128k Pittsburgh250k [1] 278k TokyoTM/247 [46] 189k MSLS [50] 1.7M San Francisco Landmark [8] 1.1M Aachen [41] 4k SF-XL (Ours) 41.2M…”

Deep Visual Geo-localization Benchmark

“…10, we show a more comprehensive set of results than in the main paper, comprising all the aggregation methods that can be attached to the different backbones using our software. As seen in the literature, GeM pooling [60] outperforms in general SPOC [5], MAC [62], R-MAC [71], RRM [39].…”

“…Over the years, a number of such methods have been proposed, from shallow pooling layers [5,62] to more complex modules [2,37]. Our framework allows to compute results with a number of them, namely SPOC [5], MAC [62], R-MAC [71], RRM [39], GeM [60], NetVLAD [2] and CRN [37]. While a complete list of results with all aggregation methods is shown in Appendix C.2, in Tab.…”