Multimedia System for Real-Time Photorealistic Nonground Modeling of 3D Dynamic Environment for Remote Control System

Abstract

Nowadays, unmanned ground vehicles (UGVs) are widely used for many applications. UGVs have sensors including multi-channel laser sensors, two-dimensional (2D) cameras, Global Positioning System receivers, and inertial measurement units (GPS-IMU). Multi-channel laser sensors and 2D cameras are installed to collect information regarding the environment surrounding the vehicle. Moreover, the GPS-IMU system is used to determine the position, acceleration, and velocity of the vehicle. This paper proposes a fast and effective method for modeling nonground scenes using multiple types of sensor data captured through a remote-controlled robot. The multi-channel laser sensor returns a point cloud in each frame. We separated the point clouds into ground and nonground areas before modeling the three-dimensional (3D) scenes. The ground part was used to create a dynamic triangular mesh based on the height map and vehicle position. The modeling of nonground parts in dynamic environments including moving objects is more challenging than modeling of ground parts. In the first step, we applied our object segmentation algorithm to divide nonground points into separate objects. Next, an object tracking algorithm was implemented to detect dynamic objects. Subsequently, nonground objects other than large dynamic ones, such as cars, were separated into two groups: surface objects and non-surface objects. We employed colored particles to model the non-surface objects. To model the surface and large dynamic objects, we used two dynamic projection panels to generate 3D meshes. In addition, we applied two processes to optimize the modeling result. First, we removed any trace of the moving objects, and collected the points on the dynamic objects in previous frames. Next, these points were merged with the nonground points in the current frame. We also applied slide window and near point projection techniques to fill the holes in the meshes. Finally, we applied texture mapping using 2D images captured using three cameras installed in the front of the robot. The results of the experiments prove that our nonground modeling method can be used to model photorealistic and real-time 3D scenes around a remote-controlled robot.

Geometric Feature Extraction of Point Cloud of Chemical Reactor Based on Dynamic Graph Convolution Neural Network

“…However, the disadvantage of the point cloud geometric feature extraction algorithm based on a triangular mesh is that the method needs to grid the point cloud data first, which is a complex process and susceptible to noise. In addition, the triangular patch cannot conform to the real surface topology of the point cloud. − The disadvantage of geometric feature extraction based on a scattered point cloud is that the point cloud data is transformed into a two-dimensional image, and the geometric feature of the point cloud is extracted by an improved two-dimensional edge detection method. In such a process, the three-dimensional point cloud data is transformed into a two-dimensional image, and then coordinate transformation is carried out.…”

Generative Adversarial Network-Based Method for Transforming Single RGB Image Into 3D Point Cloud

“…Three-dimensional (3D) point clouds are widely used in a variety of applications such as object tracking [1]- [4] and 3D scene reconstruction of both indoor [5], [6] and outdoor [7]- [14] environments. In the latter, 3D point clouds along with two-dimensional (2D) textures allow to reconstruct photorealistic 3D scenes.…”

Enhanced ground segmentation method for Lidar point clouds in human-centric autonomous robot systems

“…step that is extremely important for robot operation. Especially, ground cloud segmentation is a pre-processing step for many terrain reconstruction applications [8][9][10]. The ground segmentation result is used for recognizing objects, classifications, and feature extraction.…”