Real-Time Streaming Point Cloud Compression for 3D LiDAR Sensor Using U-Net

Abstract

Point cloud data from LiDAR sensors is the currently the basis of most L4 autonomous driving systems. Sharing and storing point clouds will also be important for future applications, such as accident investigation or V2V/V2X networks. Due to the huge volume of data involved, storing point clouds collected over long periods of time and transmitting point clouds in real-time are difficult tasks, making compression an indispensable step before storing or transmitting. Previous streaming point cloud compression methods, such as octree compression or video compression-based approaches, have difficulty compressing this data in real-time into very small volumes with low information loss. To reduce temporal redundancy efficiently and rapidly, in this paper we propose a real-time streaming point cloud data compression method using U-net. By utilizing raw packet data from LiDAR sensors, we can store 3D point cloud information losslessly in a 2D matrix, and convert streaming point cloud data into a video-like format. By designating some frames as reference frames and then using U-net to interpolate the remaining LiDAR frames, we can greatly reduce temporal redundancy. Our use of U-net was inspired by a video interpolation approach employed in another study. Noise in LiDAR data is a big issue which significantly affects network training and compression results. In this paper, we propose a padding strategy to alleviate the negative impact of this noise. As a result of these improvements, our proposed method can outperform octree compression, MPEG-based compression and our previously proposed SLAM-based compression method.

Measurement Study of Real-Time Virtual Reality Contents Streaming over IEEE 802.11ac Wireless Links

“…During the process of collecting data, streaming occurs in real time, thereby reducing any temporal redundancy. Therefore, it is demonstrated through experiments that the method mentioned in [14] is more efficient than the previously proposed Octree compression, MPEG-based compression, and SLAM-based compression methods. The above studies have introduced ways to compress data in point clouds themselves in their own distinct way.…”

“…The related research results for the measurements study of VR contents streaming over wireless networks can be classified into three main categories. First of all, in order to transfer and deliver point cloud information in real time, many research activities have been conducted for the compression of point cloud information because it involves an enormous amount of data [13,14].…”

“…The proposed algorithm in [14] aims at the use of point clouds in autonomous driving to investigate accidents or apply them to various applications. They tried to solve the limitation problem due to the data size of the pointer cloud via a deep learning-based streaming point cloud compression method that can be performed in real time.…”

Survey on Deep Learning-Based Point Cloud Compression

“…Tu et al (2019a) exploit this 2D structure to compress point cloud geometry with a convolutional Long Short-Term Memory (LSTM) neural network (Hochreiter and Schmidhuber, 1997;Shi et al, 2015) and residual coding; this is an extension of Tu et al (2017). Tu et al (2019b) take inspiration from video coding and divide dynamic point cloud frames into I-frames and B-frames (ITU-T, 2019). The B-frames are predicted with U-net (Ronneberger et al, 2015) based flow computation and interpolation extending work in Tu et al (2016).…”

An Unsupervised Optical Flow Estimation for Lidar Image Sequences

“…With the rapid development of Light detection and ranging (LiDAR) sensors, the captured sparse point clouds, which contain a set of 3D points representing the surrounding environment, have been widely used in various applications, such as, autonomous driving, augmented reality (AR), and drones. Recently, 2D LiDAR image sequences, which are generated by a projection from 3D LiDAR point clouds, are often encountered in many fields, e.g., the 3D semantic segmentation [1][2][3][4] and LiDAR point cloud compression (PCC) [5][6][7][8]. Among these works, some efforts on PCC aim to reduce the temporal redundancy of LiDAR point clouds, where the motion estimation becomes an essential issue.…”