Mapillary Planet-Scale Depth Dataset

Antequera, Manuel López; Gargallo, Pau; Hofinger, Markus; Bulò, Samuel Rota; Kuang, Yubin; Kontschieder, Peter

doi:10.1007/978-3-030-58536-5_35

Cited by 19 publications

(9 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Cross-modal matching between the image and the map requires extracting semantic information from visual cues. It has been shown that monocular depth estimation can rely on semantic cues [3] and that both tasks have a beneficial synergy [29,34]. We thus rely on monocular inference to lift semantic features to the BEV space.…”

Section: Neural Bird's-eye View Inferencementioning

confidence: 99%

SuperGlue: Learning Feature Matching With Graph Neural Networks

Sarlin

DeTone

Malisiewicz

et al. 2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

1,753

1,061

View full text Add to dashboard Cite

Humans can orient themselves in their 3D environments using simple 2D maps. Differently, algorithms for visual localization mostly rely on complex 3D point clouds that are expensive to build, store, and maintain over time. We bridge this gap by introducing OrienterNet, the first deep neural network that can localize an image with sub-meter accuracy using the same 2D semantic maps that humans use. Orienter-Net estimates the location and orientation of a query image by matching a neural Bird's-Eye View with open and globally available maps from OpenStreetMap, enabling anyone to localize anywhere such maps are available. OrienterNet is supervised only by camera poses but learns to perform semantic matching with a wide range of map elements in an end-to-end manner. To enable this, we introduce a large crowd-sourced dataset of images captured across 12 cities from the diverse viewpoints of cars, bikes, and pedestrians. OrienterNet generalizes to new datasets and pushes the state of the art in both robotics and AR scenarios. The code and trained model will be released publicly.

show abstract

Section: Neural Bird's-eye View Inferencementioning

confidence: 99%

SuperGlue: Learning Feature Matching With Graph Neural Networks

Sarlin

DeTone

Malisiewicz

et al. 2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

1,753

1,061

View full text Add to dashboard Cite

show abstract

“…The use of crowdmapping has appeared in several research endeavors [22,23] as well as industrial applications (Waymo [24], Lyft [25], Uber [26], and Google [27]). A prime example is Mapillary [28]; a company that created a crowdsourcing platform, whose main purpose is to map out every corner of the world. Maintenance of the maps is performed using computer vision and artificial intelligence (AI).…”

Section: Crowdsourced Datamentioning

confidence: 99%

Building Maps Using Monocular Image-feeds from Windshield-mounted Cameras in a Simulator Environment

Szántó¹,

Kobál²,

Vajta³

et al. 2023

Period. Polytech. Civil Eng.

View full text Add to dashboard Cite

3-dimensional, accurate, and up-to-date maps are essential for vehicles with autonomous capabilities, whose functionality is made possible by machine learning-based algorithms. Since these solutions require a tremendous amount of data for parameter optimization, simulation-to-reality (Sim2Real) methods have been proven immensely useful for training data generation. For creating realistic models to be used for synthetic data generation, crowdsourcing techniques present a resource-efficient alternative. In this paper, we show that using the Carla simulation environment, a crowdsourcing model can be created that mimics a multi-agent data gathering and processing pipeline. We developed a solution that yields dense point clouds based on monocular images and location information gathered by individual data acquisition vehicles. Our method provides scene reconstructions using the robust Structure-from-Motion (SfM) solution of Colmap. Moreover, we introduce a solution for synthesizing dense ground truth point clouds originating from the Carla simulator using a simulated data acquisition pipeline. We compare the results of the Colmap reconstruction with the reference point cloud after aligning them using the iterative closest point algorithm. Our results show that a precise point cloud reconstruction was feasible with this crowdsourcing-based approach, with 54\% of the reconstructed points having an error under 0.05 m, and a weighted root mean square error of 0.0449 m for the entire point cloud.

show abstract

“…Off-theshelf SfM algorithms have limited success in reconstructing objects from Mapillary data. However, with appropriate calibration and tuning, the Mapillary Research team and collaborators had successfully demonstrated that scene reconstruction from Mapillary data is possible [51]. The research led to the publication of a set of 750,000 Mapillary images with depth information.…”

Section: Task 2: Scene Reconstructionmentioning

confidence: 99%

“…The research led to the publication of a set of 750,000 Mapillary images with depth information. Antequera et al [51] also showed that such a dataset can be used to train single-image-depth networks to predict depth information from individual images. The methodology and data introduced in [51] will potentially be the basis for scene reconstruction in this project.…”

Section: Task 2: Scene Reconstructionmentioning

confidence: 99%

In Search of Basement Indicators from Street View Imagery Data: An Investigation of Data Sources and Analysis Strategies

Bertolotto

Ofterdinger³

et al. 2023

Künstl Intell

View full text Add to dashboard Cite

Street view imagery databases such as Google Street View, Mapillary, and Karta View provide great spatial and temporal coverage for many cities globally. Those data, when coupled with appropriate computer vision algorithms, can provide an effective means to analyse aspects of the urban environment at scale. As an effort to enhance current practices in urban flood risk assessment, this project investigates a potential use of street view imagery data to identify building features that indicate buildings’ vulnerability to flooding (e.g., basements and semi-basements). In particular, this paper discusses (1) building features indicating the presence of basement structures, (2) available imagery data sources capturing those features, and (3) computer vision algorithms capable of automatically detecting the features of interest. The paper also reviews existing methods for reconstructing geometry representations of the extracted features from images and potential approaches to account for data quality issues. Preliminary experiments were conducted, which confirmed the usability of the freely available Mapillary images for detecting basement railings as an example type of basement features, as well as geolocating the features.

show abstract

Mapillary Planet-Scale Depth Dataset

Cited by 19 publications

References 22 publications

SuperGlue: Learning Feature Matching With Graph Neural Networks

SuperGlue: Learning Feature Matching With Graph Neural Networks

Building Maps Using Monocular Image-feeds from Windshield-mounted Cameras in a Simulator Environment

In Search of Basement Indicators from Street View Imagery Data: An Investigation of Data Sources and Analysis Strategies

Contact Info

Product

Resources

About