Glidar: An OpenGL-based, Real-Time, and Open Source 3D Sensor Simulator for Testing Computer Vision Algorithms

This article describes a novel fusion of a generative formal model for three-dimensional (3D) shapes with deep learning (DL) methods to understand the geometric structure of 3D objects and the relationships between their components, given a collection of unorganized point cloud measurements. Formal 3D shape models are implemented as shape grammar programs written in Procedural Shape Modeling Language (PSML). Users write PSML programs to describe complex objects, and DL networks estimate the configured free parameters of the program to generate 3D shapes. Users write PSML programs to enforce fundamental rules that define an object class and encode object attributes, including shapes, components, size, position, etc., into a parametric representation of objects. This fusion of the generative model with DL offers artificial intelligence (AI) models an opportunity to better understand the geometric organization of objects in terms of their components and their relationships to other objects. This approach allows human-in-the-loop control over DL estimates by specifying lists of candidate objects, the shape variations that each object can exhibit, and the level of detail or, equivalently, dimension of the latent representation of the shape. The results demonstrate the advantages of the proposed method over competing approaches.

Section: Methodsmentioning

confidence: 99%

Bridging Formal Shape Models and Deep Learning: A Novel Fusion for Understanding 3D Objects

Zhang,

Willis

2024

“…The sensor point cloud simulation software is implemented with C++ codes and used in the experiments. Other point cloud simulation software can also be used such as the reference [ 31 ].…”

Section: Methodsmentioning

confidence: 99%

LiDAR-Based Non-Cooperative Tumbling Spacecraft Pose Tracking by Fusing Depth Maps and Point Clouds

Zhao

2018

How to determine the relative pose between the chaser spacecraft and the high-speed tumbling target spacecraft at close range, which is an essential step in space proximity missions, is very challenging. This paper proposes a LiDAR-based pose tracking method by fusing depth maps and point clouds. The key point is to estimate the roll angle variation in adjacent sensor data by using the line detection and matching in depth maps. The simplification of adaptive voxelized grid point cloud based on the real-time relative position is adapted in order to satisfy the real-time requirement in the approaching process. In addition, the Iterative Closest Point algorithm is used to align the simplified sparse point cloud with the known target model point cloud in order to obtain the relative pose. Numerical experiments, which simulate the typical tumbling motion of the target and the approaching process, are performed to demonstrate the method. The experimental results show that the method has capability of estimating the real-time 6-DOF relative pose and dealing with large pose variations.

“…There is no need for special interfaces or the generation of a digital twin of the simulation environment. A lidar model which can be included directly in the rendering pipeline is GLIDAR [ 14 ]. It is based on OpenGL [ 15 ] with a specific fragment shader retrieving data from the depth buffer.…”

Section: Previous Workmentioning

confidence: 99%

LiMOX—A Point Cloud Lidar Model Toolbox Based on NVIDIA OptiX Ray Tracing Engine

Rott,

Ritter,

Ladstätter

et al. 2024

Virtual testing and validation are building blocks in the development of autonomous systems, in particular autonomous driving. Perception sensor models gained more attention to cover the entire tool chain of the sense–plan–act cycle, in a realistic test setup. In the literature or state-of-the-art software tools various kinds of lidar sensor models are available. We present a point cloud lidar sensor model, based on ray tracing, developed for a modular software architecture, which can be used stand-alone. The model is highly parametrizable and designed as a toolbox to simulate different kinds of lidar sensors. It is linked to an infrared material database to incorporate physical sensor effects introduced by the ray–surface interaction. The maximum detectable range depends on the material reflectivity, which can be covered with this approach. The angular dependence and maximum range for different Lambertian target materials are studied. Point clouds from a scene in an urban street environment are compared for different sensor parameters.