HD Maps: Fine-Grained Road Segmentation by Parsing Ground and Aerial Images

Máttyus, Gellért; Wang, Shenlong; Fidler, Sanja; Urtasun, Raquel

doi:10.1109/cvpr.2016.393

Cited by 135 publications

(71 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They use a trees inventory from the city of Pasadena to validate their detection model and train a fine grained CNN based on GoogleNet [89], to perform fine-grained classification of the trees species on the detections, with impressive results. Authors of [150] take advantage of an approach that combines CNN and MRF and can estimate fine grained categories (e.g., road, sidewalk, background, building and parking) by doing joint inference over both monocular aerial imagery and ground images taken from a stereo camera on top of a car.…”

Section: Multimodal Data Fusionmentioning

confidence: 99%

Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources

Zhu

Tuia

Mou

et al. 2017

IEEE Geosci. Remote Sens. Mag.

2,439

1,204

View full text Add to dashboard Cite

This is the pre-acceptance version, to read the final version please go to IEEE Geoscience and Remote Sensing Magazine on IEEE XPlore.Standing at the paradigm shift towards data-intensive science, machine learning techniques are becoming increasingly important. In particular, as a major breakthrough in the field, deep learning has proven as an extremely powerful tool in many fields. Shall we embrace deep learning as the key to all? Or, should we resist a "black-box" solution? There are controversial opinions in the remote sensing community. In this article, we analyze the challenges of using deep learning for remote sensing data analysis, review the recent advances, and provide resources to make deep learning in remote sensing ridiculously simple to start with. More importantly, we advocate remote sensing scientists to bring their expertise into deep learning, and use it as an implicit general model to tackle unprecedented large-scale influential challenges, such as climate change and urbanization. Following this wave of success and thanks to the increased availability of data and computational resources, the use of deep learning in remote sensing is finally taking off in remote sensing as well. Remote sensing data bring some new challenges for deep learning, since satellite image analysis raises some unique questions that translate into challenging new scientific questions:• Remote sensing data are often multi-modal, e.g. from optical (multi-and hyperspectral) and synthetic aperture radar (SAR) sensors, where both the imaging geometries and the content are completely different. Data and information fusion uses these complementary

show abstract

Section: Multimodal Data Fusionmentioning

confidence: 99%

Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources

Zhu

Tuia

Mou

et al. 2017

IEEE Geosci. Remote Sens. Mag.

2,439

1,204

View full text Add to dashboard Cite

show abstract

“…Besides classical registration approaches, a variety of research studies indicate the high potential of deep learning methods for different applications in remote sensing, such as classification of hyperspectral data [25][26][27], enhancement of existing road maps [28,29], high-resolution SAR image classification [30] or pansharpening [31]. In the context of image matching, deep matching networks were successfully trained for tasks such as stereo estimation [32,33], optical flow estimation [34,35], aerial image matching [36] or ground to aerial image matching [37].…”

Section: Related Workmentioning

confidence: 99%

Exploiting Deep Matching and SAR Data for the Geo-Localization Accuracy Improvement of Optical Satellite Images

et al. 2017

Self Cite

View full text Add to dashboard Cite

Improving the geo-localization of optical satellite images is an important pre-processing step for many remote sensing tasks like monitoring by image time series or scene analysis after sudden events. These tasks require geo-referenced and precisely co-registered multi-sensor data. Images captured by the high resolution synthetic aperture radar (SAR) satellite TerraSAR-X exhibit an absolute geo-location accuracy within a few decimeters. These images represent therefore a reliable source to improve the geo-location accuracy of optical images, which is in the order of tens of meters. In this paper, a deep learning-based approach for the geo-localization accuracy improvement of optical satellite images through SAR reference data is investigated. Image registration between SAR and optical images requires few, but accurate and reliable matching points. These are derived from a Siamese neural network. The network is trained using TerraSAR-X and PRISM image pairs covering greater urban areas spread over Europe, in order to learn the two-dimensional spatial shifts between optical and SAR image patches. Results confirm that accurate and reliable matching points can be generated with higher matching accuracy and precision with respect to state-of-the-art approaches.

show abstract

“…Street view imagery has been combined with aerial imagery to achieve fine-grained road segmentation [3], land use classification [5], and tree detection/classification [4]. It is assumed that objects are discovered through both imaging modalities.…”

Section: Related Workmentioning

confidence: 99%

“…In the last decade, a considerable effort has been directed towards the intelligent use of street view imagery in multiple areas such as mapping [4,6,13], image referencing [14], vegetation monitoring [15], navigation [3], and rendering and visualization [16,17]. Street view imagery has been combined with aerial imagery to achieve fine-grained road segmentation [3], land use classification [5], and tree detection/classification [4].…”

Section: Related Workmentioning

confidence: 99%

“…Much research has been dedicated to leveraging street view imagery in combination with other data-sources such as remotely sensed imagery [3][4][5] or crowd-sourced information [6,7] to map particular types of objects or areas. Here we address the problem of automated discovery and geotagging of recurring objects using street view images as a sole source of input data.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Automatic Discovery and Geotagging of Objects from Street View Imagery

2018

View full text Add to dashboard Cite

Abstract:Many applications, such as autonomous navigation, urban planning, and asset monitoring, rely on the availability of accurate information about objects and their geolocations. In this paper, we propose the automatic detection and computation of the coordinates of recurring stationary objects of interest using street view imagery. Our processing pipeline relies on two fully convolutional neural networks: the first segments objects in the images, while the second estimates their distance from the camera. To geolocate all the detected objects coherently we propose a novel custom Markov random field model to estimate the objects' geolocation. The novelty of the resulting pipeline is the combined use of monocular depth estimation and triangulation to enable automatic mapping of complex scenes with the simultaneous presence of multiple, visually similar objects of interest. We validate experimentally the effectiveness of our approach on two object classes: traffic lights and telegraph poles. The experiments report high object recall rates and position precision of approximately 2 m, which is approaching the precision of single-frequency GPS receivers.

show abstract

HD Maps: Fine-Grained Road Segmentation by Parsing Ground and Aerial Images

Cited by 135 publications

References 21 publications

Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources

Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources

Exploiting Deep Matching and SAR Data for the Geo-Localization Accuracy Improvement of Optical Satellite Images

Automatic Discovery and Geotagging of Objects from Street View Imagery

Contact Info

Product

Resources

About