3D Urban Buildings Extraction Based on Airborne LiDAR and Photogrammetric Point Cloud Fusion According to U-Net Deep Learning Model Segmentation

Abstract

The LiDAR and photogrammetric point clouds fusion procedure for building extraction according to U-Net deep learning model segmentation is provided and tested. Firstly, an initial geolocalization process is performed for photogrammetric point clouds generated using structure-from-motion and dense-matching methods. Then, point cloud segmentation is carried out based on U-Net deep learning model. The precision of the U-Net model for buildings extraction reachs 87%, with F-score of 0.89 and IoU of 0.80. It is shown that the U-Net method is effective for high-resolution image extraction. The detailed information can accurately be identified and extracted, such as vegetation located between buildings and roads. After segmentation, each chunk of the LiDAR and photogrammetric point clouds are finely registered and merged based on the iterative closest point algorithm. Finally, the fused point clouds are obtained. It shows that the structure and shape of the buildings could be delineated from the fused point clouds when both enough ground points and a higher point density are available. Furthermore, color information improves both visualization effect and properties identification. The experiments are conducted to extract individual buildings from three types of point clouds in three plots. A DoN (Difference of Normals) approach is used to isolate 3D buildings from other objects in densely built-up areas. It shows that most building extraction results have a Precision > 0.9 and favorable Recall and F-score values. Although the LiDAR extraction results have some advantages over the photogrammetric and fused ones in terms of Precision, the Recall and F-score results appear best for the fused point clouds. It shows that the fused data contains a high point density and RGB color information and could improve the building extraction.

ResAt-UNet: A U-Shaped Network Using ResNet and Attention Module for Image Segmentation of Urban Buildings

“…A Difference of Normals (DoN) approach is used to isolate 3D buildings from other objects in densely built-up areas. It shows that most building extraction results have a Precision > 0.9 and favorable Recall and F-score values [20]. Although the above algorithm applied to urban building segmentation has achieved good segmentation results, there are still some shortcomings.…”

“…The past decade has witnessed the continuous development of neural network, which is also used in cloud segmentation [8-[13], power transmission system [14], and building segmentation [15][16][17][18][19][20], etc. In 2015, Long et al [21] proposed the Fully Convolutional Network (FCN), which was the first network successfully used for image segmentation.…”

Investigation of geometric object and indoor mapping capacity of Apple iPhone 12 Pro LiDAR

“…Development of micro-dimensions of sensors and electro-mechanical components lead to support these systems with high computational power. This digital advancement has been aided by different technology advances, resulting in a broader use for the 3D documentation process [14][15][16][17][18].…”

The Potential of Multispectral Lidar for Building Extraction in a Complex Urban Area Using CANUPO Algorithm

“…These studies include the works of [16][17][18][19][20]. [21] have made progress in the field of building extraction from lidar data. However, a challenge remains in directly extracting buildings from the raw lidar point cloud directly due to the irregularity of the cloud points and the large volume of data, in addition to the lack of proliferation of deep learning networks that deal with those points directly and such methods need specialized people in machine learning.…”