A Phase-Congruency-Based Scene Abstraction Approach for 2D-3D Registration of Aerial Optical and LiDAR Images

Abstract

Registration of aerial images to enrich 3D LIght Detection and Ranging (LiDAR) points with radiometric information can enhance the capability of object detection, scene classification, and semantic segmentation. However, airborne LiDAR data may not always come with on-board optical images collected during the same flight mission. Indirect geo-referencing can be adopted, if ancillary imagery data is found available. Nevertheless, automatic recognition of control primitives in LiDAR and imagery datasets becomes challenging, especially when they are collected on different dates. This paper proposes a generic registration mechanism based on using the Phase Congruency (PC) model and scene abstraction to overcome the stated challenges. The approach relies on the use of a PC measure to compute the image moments that determine the study scene's edges. Potential candidate points can be identified based on thresholding the image moments' values. A Shape Context Descriptor (SCD) is adopted to automatically pair symmetric candidate points to produce a final set of control points. Coordinate transformation parameters between the two datasets were estimated using a Least Squares (LS) adjustment for four registration models: first (affine), second, third order polynomials, and Direct Linear Transform (DLT) models. Datasets covering different urban landscapes were used to examine the proposed workflow. The Root Mean Square Error (RMSE) of the registration is between one to two pixels. The proposed workflow is found to be computationally efficient especially with small-sized datasets, and generic enough to be applied in registering various imagery data and LiDAR point clouds.

Fusion of Airborne LiDAR Point Clouds and Aerial Images for Heterogeneous Land-Use Urban Mapping

“…Figure 11 illustrates some of these errors featured by the displacement of sidewalks (Figure 11a) as well as misalignment of buildings and grasses (Figure 11c). The switch to the affine registration model at a spatial resolution of 40 cm along with the fusion of CED as described in [46] resulted in 3074 and 3045 candidate control points on the LiDAR and aerial images, respectively, which were matched into a set of 975 pairs of final control points. The whole processing took 14.7 h and yielded 1.78 and 1.19 pixels as model development and validation errors, respectively.…”

“…As concluded in [45,46], it is better to apply the PC model for LiDAR and imagery data geo-registration on 2D images resampled at lower spatial resolutions to avoid computationally expensive data processing and high model development error values. However, including CED with the PC filter as a scene abstraction approach significantly reduces the model development error values (from 5.74 to 1.78 pixels using the affine model) when processing the data at high spatial resolutions (40 cm).…”

“…The approach does not require data to be georeferenced, applies empirical registration models without knowing the physical model characteristics, and overcomes the problem of airborne LiDAR data not coming with onboard optical images collected during the same flight mission. Full details about the geo-registration approach are found in [46]. • Suggesting the inclusion of the CED to the PC filter for a more concise detection of geo-registration primitives.…”

“…Moreover, the phase congruency model runs on lower spatial-resolution versions of the images, which speeds up the processing and contributes more to figuring out common control points in both images. A detailed description of the geo-registration process can be found in [46].…”

“…To solve this problem, we switched to the affine registration model at a spatial resolution of 40 cm instead of the third-order polynomial at a 200 cm spatial-resolution. For more robust results, we fused the CED (Canny Edge Detector) algorithm with the geo-registration workflow for scene abstraction instead of image clustering by running it on the PC filter output, as suggested by Megahed et al [46]. We used the scikit-image library [70] to read images and detect edges, OpenCV [71] to save, and GDAL to georeference edges [72].…”