Metric Evaluation Pipeline for 3d Modeling of Urban Scenes

Abstract

ABSTRACT:Publicly available benchmark data and metric evaluation approaches have been instrumental in enabling research to advance state of the art methods for remote sensing applications in urban 3D modeling. Most publicly available benchmark datasets have consisted of high resolution airborne imagery and lidar suitable for 3D modeling on a relatively modest scale. To enable research in larger scale 3D mapping, we have recently released a public benchmark dataset with multi-view commercial satellite imagery and metrics to compare 3D point clouds with lidar ground truth. We now define a more complete metric evaluation pipeline developed as publicly available open source software to assess semantically labeled 3D models of complex urban scenes derived from multi-view commercial satellite imagery. Evaluation metrics in our pipeline include horizontal and vertical accuracy and completeness, volumetric completeness and correctness, perceptual quality, and model simplicity. Sources of ground truth include airborne lidar and overhead imagery, and we demonstrate a semi-automated process for producing accurate ground truth shape files to characterize building footprints. We validate our current metric evaluation pipeline using 3D models produced using open source multi-view stereo methods. Data and software is made publicly available to enable further research and planned benchmarking activities.

A Study of Using Fully Convolutional Network for Treetop Detection on Remote Sensing Data

“…The size of this area is 6.740 km by 6.914 km (22469 pixels × 23048 pixels) and the scene contains both forested prairies and urban areas. The satellite images in this work are from the multi-view benchmark dataset provided by John's Hopkins University Applied Physics Lab (JHUAPL) (Bosch et al, 2016;Bosch et al, 2017), containing multiple worldview2/3 images over this area across two years with a total of approximately 50 images. They were taken under various conditions containing on-track and off-track stereos with the ground resolution around 0.3 meters, a complete set of meta information can be found on their hosting website.…”

Leveraging Vision Reconstruction Pipelines for Satellite Imagery

“…Satellite imagery finds applications in a variety of domains, including ecological monitoring, 3D urban modelling, and navigation. The growing number of academic and commercial satellites, and increasing accessibility of satellite imagery, has sparked greater interest in use of satellite imagery for large-scale 3D reconstruction of the Earth's surface [25,11,4,5]. Satellites can capture the same geographic area over the course of seconds, days, months, and years, yielding a wealth of imagery available for 3D reconstruction.…”

State of the Art in Dense Image Matching Cost Computation for High-Resolution Satellite Stereo

“…In our experiments, a 1×1 km 2 area of interest with complicated ground object classes is focused on, charactered with the elevated road/bridge, low-rise residential buildings, high-rise buildings are selected. The satellite images and corresponding ground truth LiDAR data are covering this region created for the Creation of Operationally Realistic 3D Environment, CORE3D (Marc Bosch et al, 2016;M Bosch et al, 2017). WorldView-3 satellite imagery is provided courtesy of DigitalGlobe, and HSIP provides ground truth LiDAR.…”