Urban Semantic 3D Reconstruction From Multiview Satellite Imagery

Leotta, Matthew J.; Long, Chengjiang; Jacquet, Bastien; Zins, Matthieu; Lipsa, Dan; Shan, Jie; Xu, Bo; Li, Zhixin; Zhang, Xu; Chang, Shih‐Fu; Purri, Matthew; Jing, Xue; Dana, Kristin

doi:10.1109/cvprw.2019.00186

Cited by 44 publications

(38 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Computer Vision community has proposed different methods to correct the pointing error of RPC camera models. Bundle adjustment based solutions are a generally accepted practice that consist in detecting inter-image tiepoints and applying a compensating function to the original RPCs so that the back-projections of the tie-points are coincident in the 3D world [14,5,22,13,18].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

To Bundle Adjust or Not: A Comparison of Relative Geolocation Correction Strategies for Satellite Multi-View Stereo

Marí

Franchis²,

Meinhardt-Llopis³

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

The generation of up-to-date accurate 3D models from multi-view satellite images has recently become a hot research topic. A well-known challenge of this problem is to put all cameras into a common frame of reference, since depending on the satellite geopositioning equipment the camera parameters may contain errors of up to tens of meters on the ground. In this context, bundle adjustment based techniques, relying on the identification of a set of tie-points and the correction of the camera models to make them coincident, have become a generally accepted practice. However, new approaches capable of producing state-of-the-art results without the use of prior bundle adjustment have also been proposed. This work aims to compare both strategies and assess the practical impact of using bundle adjustment for 3D reconstruction from multi-view satellite images.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Our evaluation will focus on the reconstruction of small areas of interest. Indeed, the increasing availability of satellite imagery has enabled the exploitation of incidental imagery for 3D reconstruction [18]. This is useful for monitoring applications in which a concrete area needs to be reconstructed from the available imagery.…”

Section: Introductionmentioning

confidence: 99%

To Bundle Adjust or Not: A Comparison of Relative Geolocation Correction Strategies for Satellite Multi-View Stereo

Marí

Franchis²,

Meinhardt-Llopis³

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

show abstract

“…A relevant contribution is the one described in [57] that won the 2019 IEEE GRSS Data Fusion Contest for Multi-View Semantic Stereo [58]. The work in [59] also uses off-nadir WV3 images for semantic labeling. Both these approaches still treat the different views of the same scene on the ground independently during training.…”

Section: Literature Reviewmentioning

confidence: 99%

Semantic Labeling of Large-Area Geographic Regions Using Multiview and Multidate Satellite Images and Noisy OSM Training Labels

Comandur

Kak

2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

We present a novel multi-view training framework and CNN architecture for combining information from multiple overlapping satellite images and noisy training labels derived from OpenStreetMap (OSM) to semantically label buildings and roads across large geographic regions (100 km 2 ). Our approach to multi-view semantic segmentation yields a 4-7% improvement in the per-class IoU scores compared to the traditional approaches that use the views independently of one another. A unique (and, perhaps, surprising) property of our system is that modifications that are added to the tail-end of the CNN for learning from the multi-view data can be discarded at the time of inference with a relatively small penalty in the overall performance. This implies that the benefits of training using multiple views are absorbed by all the layers of the network. Additionally, our approach only adds a small overhead in terms of the GPUmemory consumption even when training with as many as 32 views per scene. The system we present is end-to-end automated, which facilitates comparing the classifiers trained directly on true orthophotos vis-a-vis first training them on the off-nadir images and subsequently translating the predicted labels to geographical coordinates. With no human supervision, our IoU scores for the buildings and roads classes are 0.8 and 0.64 respectively which are better than state-of-the-art approaches that use OSM labels and that are not completely automated.

show abstract

“…However, there is a small amount of work exploring the semantic segmentation of satellite image-derived point clouds. In their investigation, Leotta et al (2019) develop an end-to-end system for segmenting buildings and bridges from terrain, by using point clouds derived from WorldView-3 multi-view satellite imagery. A reason for the reduced research in this direction would be the low 3D quality of the obtained point clouds, caused generally by the smoothing effects of the used dense image matching algorithm, as well as the 3D reconstruction difficulties encountered in the occluded, nontextured areas, water surfaces and repetitive patterns.…”

Section: Introductionmentioning

confidence: 99%

Supervised Classification and Its Repeatability for Point Clouds From Dense VHR Tri-Stereo Satellite Image Matching Using Machine Learning

Loghin

Pfeifer

Otepka-Schremmer

2020

ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

Abstract. Image matching of aerial or satellite images and Airborne Laser Scanning (ALS) are the two main techniques for the acquisition of geospatial information (3D point clouds), used for mapping and 3D modelling of large surface areas. While ALS point cloud classification is a widely investigated topic, there are fewer studies related to the image-derived point clouds, even less for point clouds derived from stereo satellite imagery. Therefore, the main focus of this contribution is a comparative analysis and evaluation of a supervised machine learning classification method that exploits the full 3D content of point clouds generated by dense image matching of tri-stereo Very High Resolution (VHR) satellite imagery. The images were collected with two different sensors (Pléiades and WorldView-3) at different timestamps for a study area covering a surface of 24 km2, located in Waldviertel, Lower Austria. In particular, we evaluate the performance and precision of the classifier by analysing the variation of the results obtained after multiple scenarios using different training and test data sets. The temporal difference of the two Pléiades acquisitions (7 days) allowed us to calculate the repeatability of the adopted machine learning algorithm for the classification. Additionally, we investigate how the different acquisition geometries (ground sample distance, viewing and convergence angles) influence the performance of classifying the satellite image-derived point clouds into five object classes: ground, trees, roads, buildings, and vehicles. Our experimental results indicate that, in overall the classifier performs very similar in all situations, with values for the F1-score between 0.63 and 0.65 and overall accuracies beyond 93%. As a measure of repeatability, stable classes such as buildings and roads show a variation below 3% for the F1-score between the two Pléiades acquisitions, proving the stability of the model.

show abstract

Urban Semantic 3D Reconstruction From Multiview Satellite Imagery

Cited by 44 publications

References 27 publications

To Bundle Adjust or Not: A Comparison of Relative Geolocation Correction Strategies for Satellite Multi-View Stereo

To Bundle Adjust or Not: A Comparison of Relative Geolocation Correction Strategies for Satellite Multi-View Stereo

Semantic Labeling of Large-Area Geographic Regions Using Multiview and Multidate Satellite Images and Noisy OSM Training Labels

Supervised Classification and Its Repeatability for Point Clouds From Dense VHR Tri-Stereo Satellite Image Matching Using Machine Learning

Contact Info

Product

Resources

About