Registration of Panoramic/Fish-Eye Image Sequence and LiDAR Points Using Skyline Features

Zhu, Na; Jia, Yongxing; Ji, Shunping

doi:10.3390/s18051651

Cited by 15 publications

(17 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fig. 2 and 3 show two different road scenes dataset, the panoramic cameras are Ladybug5 (Zhu et al, 2018). Figure 2 shows 10 panoramic image sequence (4000×8000 pixels), the interval between adjacent images is approximately 7 m. The length and height difference of the road are approximately 250 m and 4.6 m, which includes 3.8 million points.…”

Section: Methodsmentioning

confidence: 99%

“…Eq. (1) shows the panoramic camera model using the spherical projection, the Y axis is perpendicular to the image plane (XOZ) (Zhu et al, 2018). 1where, ( ) are the row and column coordinate of panoramic image, , is the rotation angle matrix, (x y z) are the coordinate of an object, D is used to convert the radians and image sizes (unit: pixels/rad).…”

Section: Pa-rommentioning

confidence: 99%

“…There are many registration methods for 2D and 3D data, which can be divided into 2D-2D (Miled et al, 2016) (Plötz and Roth, 2017), 2D-3D (Liu and Stamos, 2012) (Shao et al, 2017) and 3D-3D (Corsini et al, 2013) (Zheng et al, 2013). The panoramic image is a special optical image (Zhu et al, 2018) (Zhu, 2019), the existing methods for registration of MMS LiDAR points and panoramic image sequence mostly rely on GPS/IMU (Original registration method) (Yao et al, 2017). The Original registration method directly obtains the position and orientation parameters from GPS/IMU, but the accuracy is affected by dropouts, etc., which lead to unreliable registration result (Miled et al, 2016).…”

Section: Introductionmentioning

confidence: 99%

“…Li et al (Li et al, 2018) extracted vehicles from panoramic images (2048×4096 pixels) via Faster-RCNN and took it as primitive pair, then particle swarm optimization was utilized to refine the translation parameters, the registration error less than 3 pixels; however, this method relies on the parked vehicles, the application is limited. Zhu et al (Zhu et al, 2018) proposed using skyline to automatically register LiDAR points and panoramic image sequence, the brute force optimization was used to match skyline pixels and points. The mean registration error of 5 panoramic images (4000 × 8000 pixels) is approximately 10 pixels; however, the skyline pixels could represent very distant objects, while skyline points are missing in corresponding area, it may cause the mismatch of skyline pixels and points.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

REGISTRATION OF MMS LiDAR POINTS AND PANORAMIC IMAGE SEQUENCE USING RELATIVE ORIENTATION MODEL

Zhu

Wang

Jia

2020

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

Abstract. We propose using the relative orientation model (ROM) of panoramic to register the MMS LiDAR points and panoramic image sequence, which has the wide applicability. The feature points, extracted and matched from panoramic image pairs, are used to solve the relative position and attitude parameters in the ROM, then, combining the absolute position and attitude parameters of the initial panoramic image, the MMS LiDAR points and panoramic image sequence are registered. First, we propose the position/attitude ROM (PA-ROM) and attitude ROM (A-ROM) of panoramic images respectively, which are apply to the position/attitude parameters both unknown and only the attitude parameters unknown. Second, we automatically extract and match feature points from panoramic image pairs using the SURF algorithm, as these mismatching points will affect the registration accuracy, the RANSAC algorithm and ROM were used to choose the best matching points automatically. Finally, we select the feature points manually from MMS LiDAR points and panoramic image sequence as the checkpoints, and then compare the registration accuracy of continuous/discontinuous panoramic image pairs. The results show that MMS LiDAR points and panoramic image sequence are registered accurately based on ROM (7.36 and 3.75 pixels in dataset I and II), what's more, our registration method just tackle the image pairs (uninvolved LiDAR points), so it is suitable for more road scenes.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Pa-rommentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

REGISTRATION OF MMS LiDAR POINTS AND PANORAMIC IMAGE SEQUENCE USING RELATIVE ORIENTATION MODEL

Zhu

Wang

Jia

2020

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

show abstract

“…For the first category, its feature is the single pixel feature, such as ground control point (GCP). Its main procedures include feature detection, feature matching, transform model estimation and image warping [7,8,9,10]. Among these procedures, feature detection and feature matching are two core steps, and they are also two steps to which researchers devote much effort [11].…”

Section: Introductionmentioning

confidence: 99%

Eliminating the Effect of Image Border with Image Periodic Decomposition for Phase Correlation Based Remote Sensing Image Registration

Dong

Jiao

Long

et al. 2019

Sensors

View full text Add to dashboard Cite

In the remote sensing community, accurate image registration is the prerequisite of the subsequent application of remote sensing images. Phase correlation based image registration has drawn extensive attention due to its high accuracy and high efficiency. However, when the Discrete Fourier Transform (DFT) of an image is computed, the image is implicitly assumed to be periodic. In practical application, it is impossible to meet the periodic condition that opposite borders of an image are alike, and image always shows strong discontinuities across the frame border. The discontinuities cause a severe artifact in the Fourier Transform, namely the known cross structure composed of high energy coefficients along the axes. Here, this phenomenon was referred to as effect of image border. Even worse, the effect of image border corrupted its registration accuracy and success rate. Currently, the main solution is blurring out the border of the image by weighting window function on the reference and sensed image. However, the approach also inevitably filters out non-border information of an image. The existing understanding is that the design of window function should filter as little information as possible, which can improve the registration success rate and accuracy of methods based on phase correlation. In this paper, another approach of eliminating the effect of image border is proposed, namely decomposing the image into two images: one being the periodic image and the other the smooth image. Replacing the original image by the periodic one does not suffer from the effect on the image border when applying Fourier Transform. The smooth image is analogous to an error image, which has little information except at the border. Extensive experiments were carried out and showed that the novel algorithm of eliminating the image border can improve the success rate and accuracy of phase correlation based image registration in some certain cases. Additionally, we obtained a new understanding of the role of window function in eliminating the effect of image border, which is helpful for researchers to select the optimal method of eliminating the effect of image border to improve the registration success rate and accuracy.

show abstract

Automatic targetless LiDAR–camera calibration: a survey

Xiao

Wang³

et al. 2022

Artif Intell Rev

View full text Add to dashboard Cite

The recent trend of fusing complementary data from LiDARs and cameras for more accurate perception has made the extrinsic calibration between the two sensors critically important. Indeed, to align the sensors spatially for proper data fusion, the calibration process usually involves estimating the extrinsic parameters between them. Traditional LiDAR-camera calibration methods often depend on explicit targets or human intervention, which can be prohibitively expensive and cumbersome. Recognizing these weaknesses, recent methods usually adopt the autonomic targetless calibration approach, which can be conducted at a much lower cost. This paper presents a thorough review of these automatic targetless LiDAR-camera calibration methods. Specifically, based on how the potential cues in the environment are retrieved and utilized in the calibration process, we divide the methods into four categories: information theory based, feature based, ego-motion based, and learning based methods. For each category, we provide an in-depth overview with insights we have gathered, hoping to serve as a potential guidance for researchers in the related fields.

show abstract

Registration of Panoramic/Fish-Eye Image Sequence and LiDAR Points Using Skyline Features

Cited by 15 publications

References 41 publications

REGISTRATION OF MMS LiDAR POINTS AND PANORAMIC IMAGE SEQUENCE USING RELATIVE ORIENTATION MODEL

REGISTRATION OF MMS LiDAR POINTS AND PANORAMIC IMAGE SEQUENCE USING RELATIVE ORIENTATION MODEL

Eliminating the Effect of Image Border with Image Periodic Decomposition for Phase Correlation Based Remote Sensing Image Registration

Automatic targetless LiDAR–camera calibration: a survey

Contact Info

Product

Resources

About