PCR-CG: Point Cloud Registration via Deep Explicit Color and Geometry

Zhang, Yu; Yu, Junle; Huang, Xiaolin; Zhou, Wenhui; Hou, Ji

doi:10.1007/978-3-031-20080-9_26

Cited by 14 publications

(5 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Chen et al [46] address the no explicit loop closure in LiDAR SLAM by organizing data frames into training batches and alleviating the no local registration and slow convergence in global registration by proposing a novel self-supervised local to global point consistency loss. Zhang et al [47] proposed a 2D-3D cross-modality learning algorithm that uses the color information from the camera to assist the point cloud registration. Incorporating this module into the Predator [48] can significantly improve.…”

Section: Related Workmentioning

confidence: 99%

Online LiDAR-camera extrinsic parameters self-checking and recalibration

Wei,

Yan,

You

et al. 2024

Meas. Sci. Technol.

View full text Add to dashboard Cite

During production, smart cars are equipped with calibrated LiDARs and cameras. However, due to the vibrations that inevitably occur during driving, the sensors' extrinsic parameters may change slightly over time. It is a significant challenge to ensure the ongoing security of these systems throughout the car's lifetime. To address this issue, we propose a self-checking and recalibration algorithm that can continuously detect the sensor data of intelligent vehicles. If the sensor's miscalibration is detected, the data can be repaired promptly to ensure the vehicle's reliability.Our self-checking algorithm extracts features from the point cloud and image and performs pixel-wise comparisons. To improve feature quality, we utilize the patch-wise transformer to enhance local information exchange, which also benefits the subsequent extrinsic recalibration. To facilitate the study, we generate two customized datasets from the KITTI dataset and the Waymo Open Dataset. The experiments conducted on these datasets demonstrate the effectiveness of our proposed method in accurately calibrating the LiDAR and camera systems throughout the car's lifetime.This study is the first to highlight the importance of continually checking the calibrated extrinsic parameters for autonomous driving. Our findings contribute to the broader goal of improving safety and reliability in autonomous driving systems.The dataset and code are available at https://github.com/OpenCalib/LiDAR2camera_self-check.

show abstract

Section: Related Workmentioning

confidence: 99%

Online LiDAR-camera extrinsic parameters self-checking and recalibration

Wei,

Yan,

You

et al. 2024

Meas. Sci. Technol.

View full text Add to dashboard Cite

show abstract

“…Many state-ofthe-art deep learning registration methods rely solely on geometry information, neglecting texture information. However, some exceptions exist where these methods rely on intermediate media such as RGBD images, projection images, or depth maps [60][61][62]. Since deep learning methods typically process only relative positions of points, they lack colour information, which limits their applications.…”

Section: Fig 1 Visualisation Of the Point Clouds Registration Processmentioning

confidence: 99%

Fast adaptive multimodal feature registration (FAMFR): an effective high-resolution point clouds registration workflow for cultural heritage interiors

Foryś,

Sitnik,

Markiewicz

et al. 2023

Herit Sci

View full text Add to dashboard Cite

Accurate registration of 3D scans is crucial in creating precise and detailed 3D models for various applications in cultural heritage. The dataset used in this study comprised numerous point clouds collected from different rooms in the Museum of King Jan III’s Palace in Warsaw using a structured light scanner. Point clouds from three relatively small rooms at Wilanow Palace: The King’s Chinese Cabinet, The King’s Wardrobe, and The Queen’s Antecabinet exhibit intricate geometric and decorative surfaces with diverse colour and reflective properties. As a result, creating a high-resolution full 3D model require a complex and time-consuming registration process. This process often consists of several steps: data preparation, registering point clouds, final relaxation, and evaluation of the resulting model. Registering two-point clouds is the most fundamental part of this process; therefore, an effective registration workflow capable of precisely registering two-point clouds representing various cultural heritage interiors is proposed in this paper. Fast Adaptive Multimodal Feature Registration (FAMFR) workflow is based on two different handcrafted features, utilising the colour and shape of the object to accurately register point clouds with extensive surface geometry details or geometrically deficient but with rich colour decorations. Furthermore, this work emphasises the challenges associated with high-resolution point clouds registration, providing an overview of various registration techniques ranging from feature-based classic approaches to new ones based on deep learning. A comparison shows that the algorithm explicitly created for this data achieved much better results than traditional feature-based or deep learning methods by at least 35%.

show abstract

“…Current large-scale RGB-D datasets [1,4,10,34,35] provide an opportunity to learn key geometric and structural priors to provide more informed reasoning about the scale and cir-cumvent view-dependent effects, which can provide more efficient representation learning. In 3D, various successful methods have been leveraging the RGB-D datasets for constrastive point discrimination [6,24,39,43] for downstream 3D tasks, including high-level scene understanding tasks as well as low-level point matching tasks [15,42]. However, the other direction from 3D to 2D is less explored.…”

Section: Introductionmentioning

confidence: 99%

Mask3D: Pre-training 2D Vision Transformers by Learning Masked 3D Priors

Hou¹,

Dai²,

He³

et al. 2023

Preprint

View full text Add to dashboard Cite

Figure 1. We present Mask3D, which learns to embed 3D priors to 2D representations for image understanding tasks, based on a selfsupervised pre-training formulation from single RGB-D views, without requiring any camera pose or multi-view correspondence information. Our pre-training takes masked RGB and depth patches as input to reconstruct the dense depth map, and the pre-trained color backbone is used to fine-tune various downstream image understanding tasks. This results in effective ViT pre-training for a variety of downstream tasks and datasets.

show abstract

PCR-CG: Point Cloud Registration via Deep Explicit Color and Geometry

Cited by 14 publications

References 40 publications

Online LiDAR-camera extrinsic parameters self-checking and recalibration

Online LiDAR-camera extrinsic parameters self-checking and recalibration

Fast adaptive multimodal feature registration (FAMFR): an effective high-resolution point clouds registration workflow for cultural heritage interiors

Mask3D: Pre-training 2D Vision Transformers by Learning Masked 3D Priors

Contact Info

Product

Resources

About