LiDAR-Aided Interior Orientation Parameters Refinement Strategy for Consumer-Grade Cameras Onboard UAV Remote Sensing Systems

Abstract

Unmanned aerial vehicles (UAVs) are quickly emerging as a popular platform for 3D reconstruction/modeling in various applications such as precision agriculture, coastal monitoring, and emergency management. For such applications, LiDAR and frame cameras are the two most commonly used sensors for 3D mapping of the object space. For example, point clouds for the area of interest can be directly derived from LiDAR sensors onboard UAVs equipped with integrated global navigation satellite systems and inertial navigation systems (GNSS/INS). Imagery-based mapping, on the other hand, is considered to be a cost-effective and practical option and is often conducted by generating point clouds and orthophotos using structure from motion (SfM) techniques. Mapping with photogrammetric approaches requires accurate camera interior orientation parameters (IOPs), especially when direct georeferencing is utilized. Most state-of-the-art approaches for determining/refining camera IOPs depend on ground control points (GCPs). However, establishing GCPs is expensive and labor-intensive, and more importantly, the distribution and number of GCPs are usually less than optimal to provide adequate control for determining and/or refining camera IOPs. Moreover, consumer-grade cameras with unstable IOPs have been widely used for mapping applications. Therefore, in such scenarios, where frequent camera calibration or IOP refinement is required, GCP-based approaches are impractical. To eliminate the need for GCPs, this study uses LiDAR data as a reference surface to perform in situ refinement of camera IOPs. The proposed refinement strategy is conducted in three main steps. An image-based sparse point cloud is first generated via a GNSS/INS-assisted SfM strategy. Then, LiDAR points corresponding to the resultant image-based sparse point cloud are identified through an iterative plane fitting approach and are referred to as LiDAR control points (LCPs). Finally, IOPs of the utilized camera are refined through a GNSS/INS-assisted bundle adjustment procedure using LCPs. Seven datasets over two study sites with a variety of geomorphic features are used to evaluate the performance of the developed strategy. The results illustrate the ability of the proposed approach to achieve an object space absolute accuracy of 3–5 cm (i.e., 5–10 times the ground sampling distance) at a 41 m flying height.

Assessing Vehicle Profiling Accuracy of Handheld LiDAR Compared to Terrestrial Laser Scanning for Crash Scene Reconstruction

“…Once the point clouds from handheld LiDAR and TLS (FARO, Trimble) are registered, the discrepancy between the two point clouds is estimated by calculating the cloud-to-cloud distance using the strategy proposed in [ 18 ], as illustrated in Figure 8 . For a given point in the source point cloud (blue point in Figure 8 a), its closest point in the reference point cloud is first identified, as shown by the green point in Figure 8 b.…”

New Orthophoto Generation Strategies from UAV and Ground Remote Sensing Platforms for High-Throughput Phenotyping

“…Aside from inaccurate system calibration and georeferencing parameters, factors that would impact the quality of derived orthophotos include (a) imprecise DSM, (b) pixilation and double mapping artifacts, and (c) seamline distortions. Several research efforts have been conducted, and proved successful, to improve the georeferencing and system calibration parameters of imaging systems onboard remote sensing platforms [14][15][16][17]. Imprecise DSM problems would be more pronounced when dealing with large-scale imagery over a complex object space, which is the key characteristic of late-season imaging using UAV and ground platforms over breeding trials with varying genotypes in neighboring plots (Figure 1a).…”

“…The system calibration includes the IOP estimation and evaluation of the mounting parameters (i.e., relative position and orientation) between cameras and the GNSS/INS unit. In this study, the cameras' IOPs are estimated and refined through calibration procedures proposed in previous studies [17,44]. The USGS Simultaneous Multi-Frame Analytical Calibration (SMAC) distortion model-which encompasses the principal distance c, principal point coordinates (x p , y p ), and radial and decentering lens distortion coefficients (K 1 , K 2 , P 1 , P 2 )-is adopted.…”

Multiscale Image Matching for Automated Calibration of UAV-Based Frame and Line Camera Systems

“…Gneeniss et al [23] developed an in-flight camera IOP refinement procedure using a least squares surface matching algorithm to align SfM-derived point clouds to LiDAR control points (LCPs). In a similar study, Zhou et al [24] used photogrammetric points as well as LiDAR data for refining camera parameters without the need for GCPs. However, there is still a gap when it comes to automated spatial and temporal calibration for frame and line camera systems without the need for GCPs.…”