A New Framework for Registration of Semantic Point Clouds from Stereo and RGB-D Cameras

Zhang, Ray; Lin, Tai-Chia; Lin, Chien Erh; Parkison, Steven A.; Clark, W. Crawford; Grizzle, Jessy W.; Eustice, Ryan M.; Ghaffari, Maani

doi:10.1109/icra48506.2021.9561929

Cited by 11 publications

(7 citation statements)

References 126 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Illustrated in Figure 1 , the following formulation provides a general framework for lifting semantically labeled point clouds into a function space to solve a registration problem ( Ghaffari et al, 2019 ; Clark et al, 2021; Zhang et al, 2021 ). Consider two (finite) collections of points, X = { x i },

.…”

Section: Rkhs Registration For Spatial-semantic Perceptionmentioning

confidence: 99%

“… Point clouds X and Z are represented by two continuous functions f X , f Z in an RKHS. Each point x i has its own semantic labels, ℓ X ( x i ), encoded in the corresponding function representation via a tensor product representation ( Zhang et al, 2021 ). The registration is formulated as maximizing the inner product between two point cloud functions.…”

Section: Rkhs Registration For Spatial-semantic Perceptionmentioning

confidence: 99%

“…Section 2 presents a nonparametric analytical framework that models semantically labeled point clouds for solving the sensor registration problem Clark et al, 2021;Zhang et al, 2021). The framework lifts the data into a Reproducing Kernel Hilbert Space (RKHS), where the inner product structure captures the cross-correlation between two labeled point clouds as functions.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Progress in symmetry preserving robot perception and control through geometry and learning

et al. 2022

Self Cite

View full text Add to dashboard Cite

This article reports on recent progress in robot perception and control methods developed by taking the symmetry of the problem into account. Inspired by existing mathematical tools for studying the symmetry structures of geometric spaces, geometric sensor registration, state estimator, and control methods provide indispensable insights into the problem formulations and generalization of robotics algorithms to challenging unknown environments. When combined with computational methods for learning hard-to-measure quantities, symmetry-preserving methods unleash tremendous performance. The article supports this claim by showcasing experimental results of robot perception, state estimation, and control in real-world scenarios.

show abstract

.…”

Section: Rkhs Registration For Spatial-semantic Perceptionmentioning

confidence: 99%

Section: Rkhs Registration For Spatial-semantic Perceptionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Progress in symmetry preserving robot perception and control through geometry and learning

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…For example, it may fail to provide reasonable results on some outdoor sequences of the Hilti dataset [55], in which the scenarios are nearly empty without clear human-made structures. To address this issue, resulting from insufficient point-to-surfel constraints from LiDAR data, some ideas from advanced point cloud registration methods [57] could be leveraged, such as Sdrsac [58], Teaser++ [59,60], OPRANSAC [61], CVO [62], and deep-learning-based methods [63].…”

Section: E Remarksmentioning

confidence: 99%

Observability-Aware Intrinsic and Extrinsic Calibration of LiDAR-IMU Systems

Lv¹,

Zuo²,

Hu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Accurate and reliable sensor calibration is essential to fuse LiDAR and inertial measurements, which are usually available in robotic applications. In this paper, we propose a novel LiDAR-IMU calibration method within the continuoustime batch-optimization framework, where the intrinsics of both sensors and the spatial-temporal extrinsics between sensors are calibrated without using calibration infrastructure such as fiducial tags. Compared to discrete-time approaches, the continuoustime formulation has natural advantages for fusing high rate measurements from LiDAR and IMU sensors. To improve efficiency and address degenerate motions, two observabilityaware modules are leveraged: (i) The information-theoretic data selection policy selects only the most informative segments for calibration during data collection, which significantly improves the calibration efficiency by processing only the selected informative segments. (ii) The observability-aware state update mechanism in nonlinear least-squares optimization updates only the identifiable directions in the state space with truncated singular value decomposition (TSVD), which enables accurate calibration results even under degenerate cases where informative data segments are not available. The proposed LiDAR-IMU calibration approach has been validated extensively in both simulated and real-world experiments with different robot platforms, demonstrating its high accuracy and repeatability in commonly-seen human-made environments. We also open source our codebase to benefit the research community: https://github.com/APRIL-ZJU/OA-LICalib.

show abstract

“…V-A is circumvented. CVO [33,34] represents a point cloud as a function in a reproducing kernel Hilbert space, transforming the registration problem to maximizing the inner product of two functions. A series of deep-learning-based methods attempt to extract a global feature embedding for a point cloud and solve the registration by aligning the global features.…”

Section: B Correspondence-free Point Cloud Registrationmentioning

confidence: 99%

Correspondence-Free Point Cloud Registration with SO(3)-Equivariant Implicit Shape Representations

Zhu¹,

Ghaffari²,

Peng

2021

Preprint

Self Cite

View full text Add to dashboard Cite

This paper proposes a correspondence-free method for point cloud rotational registration. We learn an embedding for each point cloud in a feature space that preserves the SO(3)-equivariance property, enabled by recent developments in equivariant neural networks. The proposed shape registration method achieves three major advantages through combining equivariant feature learning with implicit shape models. First, the necessity of data association is removed because of the permutation-invariant property in network architectures similar to PointNet. Second, the registration in feature space can be solved in closed-form using Horn's method due to the SO(3)equivariance property. Third, the registration is robust to noise in the point cloud because of implicit shape learning. The experimental results show superior performance compared with existing correspondence-free deep registration methods.

show abstract

A New Framework for Registration of Semantic Point Clouds from Stereo and RGB-D Cameras

Cited by 11 publications

References 126 publications

Progress in symmetry preserving robot perception and control through geometry and learning

Progress in symmetry preserving robot perception and control through geometry and learning

Observability-Aware Intrinsic and Extrinsic Calibration of LiDAR-IMU Systems

Correspondence-Free Point Cloud Registration with SO(3)-Equivariant Implicit Shape Representations

Contact Info

Product

Resources

About