Revisiting IM2GPS in the Deep Learning Era

Abstract

Image geolocalization, inferring the geographic location of an image, is a challenging computer vision problem with many potential applications. The recent state-of-the-art approach to this problem is a deep image classification approach in which the world is spatially divided into cells and a deep network is trained to predict the correct cell for a given image. We propose to combine this approach with the original Im2GPS approach in which a query image is matched against a database of geotagged images and the location is inferred from the retrieved set. We estimate the geographic location of a query image by applying kernel density estimation to the locations of its nearest neighbors in the reference database. Interestingly, we find that the best features for our retrieval task are derived from networks trained with classification loss even though we do not use a classification approach at test time. Training with classification loss outperforms several deep feature learning methods (e.g. Siamese networks with contrastive of triplet loss) more typical for retrieval applications. Our simple approach achieves state-of-the-art geolocalization accuracy while also requiring significantly less training data.

Hardness-Aware Deep Metric Learning

“…They take advantage of deep neural networks [17,27,31,11] to construct a mapping from the data space to the embedding space so that the Euclidean distance in the embedding space can reflect the actual semantic distance between data points, i.e., a relatively large distance between inter-class samples and a relatively small distance between intra-class samples. Recently a variety of deep metric learning methods have been proposed and have demonstrated strong effectiveness in various tasks, such as image retrieval [30,23,19,5], person re-identification [26,37,48,2], and geo-localization [35,14,34]. Figure 1.…”

Stochastic Attraction-Repulsion Embedding for Large Scale Image Localization

“…8 shows the results of the quantitative comparison between our method and other deep metric learning methods. Our theoretically-grounded method outperforms the Contrastive loss [25] and Geo-classification loss [40], while remains comparable with other state-of-the-art methods. [12] 69.20 ----…”

“…Although deep metric-learning methods have shown their effectiveness in classification and fine-grain recognition tasks, their abilities in the IBL task are unknown. As another contribution of this paper, we show the performances of six current state-of-the-art deep metric-learning methods in IBL, and compare our method with : (1) Contrastive loss used by [25]; (2) Lifted structure embedding [22]; (3) N-pair loss [36]; (4) N-pair angular loss [41]; (5) Geo-classification loss [40]; (6) Ratio loss [10]. Fig.…”

“…Each image is associated with a GPS-tag giving its approximate geographic location, which can be used to identify nearby images not necessarily depicting the same landmark. We follow [2,40] to identify the positive and negative images for each query image. For each query image, the positive image is the closest neighbor in the feature embedding space at its nearby geoposition, and the negatives are far away images.…”

“…Geographic classification loss For the geographic classification method [40], we use the Pitts250k-train dataset for training. We first partition the 2D geographic space into square cells, with each cell size at 25m.…”