A novel way to organize 3D LiDAR point cloud as 2D depth map height map and surface normal map

Abstract

In this paper we focus on what meaningful 2D perceptual information we can get from 3D LiDAR point cloud. Current work [1] [2] [3] have demonstrated that the depth, height and local surface normal value of a 3D data are useful features for improving Deep Neural Networks (DNNs) based object detection. We thus propose to organise LiDAR point as three different maps: dense depth map, height map and surface normal map. Specifically, given a pair of RGB image and sparse depth map projected from LiDAR point cloud, we propose a parameter self-adaptive method to upgrade sparse depth map to dense depth map, which is then passed to a convex optimisation framework to gain global enhancement. Height map is obtained by reprojecting each pixel in dense depth map into 3D coordinate, which enables us to record its height value, surface normal map is obtained by a trilateral filter constructed from depth map and RGB image. Finally, we validate our framework on both KITTI tracking dataset and Middlebury dataset 1 . To the best of our knowledge, we are the first to interpret 3D LiDAR point cloud as various 2D features and hope it will motivate more research on object detection by combing RGB image and 3D LiDAR point cloud.

Transforming a 3-D LiDAR Point Cloud Into a 2-D Dense Depth Map Through a Parameter Self-Adaptive Framework

“…Comparing with our previous work PREV [34], our proposed framework performs better in all objects and occlusion levels. We do not give the results of OurSDGE, OurSE, and OurSEGE as they produce very similar results to OurSD.…”

“…Visual comparison of depth upsampling on the KITTI data set. In each subfigure from top to bottom: RGB image superimposed with point cloud image (point cloud is bolded to be seen more clearly), JOINT [29], TGVL [27], FILTER [26], GEO [28], PREV [34], and OurSD. Note that the other three methods' results produce blurry boundaries.…”

“…Each scene includes two color images and a disparity image that serves as the depth ground truth. We compared our algorithm with a MRF optimization based method (JOINT) [29], local filter based method (FILTER) [26], global optimization based method (TGVL) [27], and geodesic distance based method (GEO) [28] and our previous work (PREV) [34]. We either use publicly released code or implemented the methods by ourselves.…”

“…Our previous work [33], [34] has attempted to involve anisotropic diffusion tensor and spatial distance, color metrics to predict the depth value. For example, in [33], we measure each depth-known pixels contribution in a color-spatial-tensor Gaussian domain, in [34], we relax the metrics to be selfadaptive and propose the parameters to be self-selective in depth-color-spatial domain.…”

“…For example, in [33], we measure each depth-known pixels contribution in a color-spatial-tensor Gaussian domain, in [34], we relax the metrics to be selfadaptive and propose the parameters to be self-selective in depth-color-spatial domain. These work both received large improvements on both indoor and outdoor depth upsampling tasks.…”