GIS-Assisted Object Detection and Geospatial Localization

Abstract

Abstract. Geographical Information System (GIS) databases contain information about many objects, such as traffic signals, road signs, fire hydrants, etc. in urban areas. This wealth of information can be utilized for assisting various computer vision tasks. In this paper, we propose a method for improving object detection using a set of priors acquired from GIS databases. Given a database of object locations from GIS and a query image with metadata, we compute the expected spatial location of the visible objects in the image. We also perform object detection in the query image (e.g., using DPM) and obtain a set of candidate bounding boxes for the objects. Then, we fuse the GIS priors with the potential detections to find the final object bounding boxes. To cope with various inaccuracies and practical complications, such as noisy metadata, occlusion, inaccuracies in GIS, and poor candidate detections, we formulate our fusion as a higher-order graph matching problem which we robustly solve using RANSAC. We demonstrate that this approach outperforms well established object detectors, such as DPM, with a large margin. Furthermore, we propose that the GIS objects can be used as cues for discovering the location where an image was taken. Our hypothesis is based on the idea that the objects visible in one image, along with their relative spatial location, provide distinctive cues for the geo-location. In order to estimate the geo-location based on the generic objects, we perform a search on a dense grid of locations over the covered area. We assign a score to each location based on the similarity of its GIS objects and the imperfect object detections in the image. We demonstrate that over a broad urban area of >10 square kilometers, this semantic approach can significantly narrow down the localization search space, and occasionally, even find the correct location.

Geo-localization using Volumetric Representations of Overhead Imagery

“…3. Impressive recent work by Ardeshir et al (2014) exploits GIS databases that contain geolocations of small scale objects, such as fire hydrants, traffic signals, trash cans, etc. to generate densely sampled hypothesized ground projections from query image view point.…”

Automatic Discovery and Geotagging of Objects from Street View Imagery

“…The GSV imagery is employed in [16,17] in conjunction with social media, such as Twitter, to perform visualization and 3D rendering. Methods proposed in [22,23] target the inverse problem to the one addressed in this work: the street view camera position is inferred from access to geolocations of stationary objects. Much work has been done on monitoring and mapping from mobile LiDAR [24][25][26] and unmanned aerial vehicle (drones) [27,28].…”

Geo-semantic segmentation

“…We can define G i as the set of 3D GPS coordinates belonging to the i th semantic segment in the GIS database which is in the field of view of the camera (in our case either a specific building or street) 1 , and g i as the two dimensional locations of the i th set of point in the image plane. A point from GIS will be projected as a vertical line on the image plane(as shown in figure 4).…”

“…In the context of utilizing GIS information, different computer vision applications have been proposed. Authors in [11,12] used GIS for registering aerial images using semantic segments and [1] leveraged GIS to improve the performance of object detection; however, to the best of our knowledge, leveraging GIS for the purpose of performing semantic/geo-semantic segmentation has not been pursued.…”