LiDAR-Flow: Dense Scene Flow Estimation from Sparse LiDAR and Stereo Images

Abstract

We propose a new approach called LiDAR-Flow to robustly estimate a dense scene flow by fusing a sparse LiDAR with stereo images. We take the advantage of the high accuracy of LiDAR to resolve the lack of information in some regions of stereo images due to textureless objects, shadows, ill-conditioned light environment and many more. Additionally, this fusion can overcome the difficulty of matching unstructured 3D points between LiDAR-only scans. Our LiDAR-Flow approach consists of three main steps; each of them exploits LiDAR measurements. First, we build strong seeds from LiDAR to enhance the robustness of matches between stereo images. The imagery part seeks the motion matches and increases the density of scene flow estimation. Then, a consistency check employs LiDAR seeds to remove the possible mismatches. Finally, LiDAR measurements constraint the edge-preserving interpolation method to fill the remaining gaps. In our evaluation we investigate the individual processing steps of our LiDAR-Flow approach and demonstrate the superior performance compared to image-only approach.• We propose LiDAR-Flow -the fusion of sparse LiDAR and stereo images for scene flow estimation.

DLOAM: Real-time and Robust LiDAR SLAM System Based on CNN in Dynamic Urban Environments

“…[14] extracted point cloud semantic features through CNN, and integrated the semantics into surfel to filter out moving objects, similarly, [18][19] use CNN for semantic segmentation to detect moving objects. [16] proposes a method called LiDAR-flow, which provides robust estimation of dense scene flows by fusing sparse LiDAR with stereo images. [17] and [20] fusing temporal information to detect dynamic objects, they use multi frame point cloud as input to regress the motion behavior of objects on the aerial view through the network, the advantage of this kind of method is to detect all moving objects in the field of vision of LiDAR, including objects not seen in the training set, which is of great significance to the safety of robotics automatic driving.…”

CamLiFlow: Bidirectional Camera-LiDAR Fusion for Joint Optical Flow and Scene Flow Estimation

“…Camera-LiDAR Fusion Cameras and LiDARs have complementary characteristics, facilitating many computer vision tasks, such as depth estimation [13,30,55], scene flow estimation [2,41], 3D object detection [10,27,36,45,51], etc. Some researchers [2,36,45,55] build a modular network and perform result-level fusion, while the others [13,27,30,41,51] explore feature-level fusion schemes including early-fusion and late-fusion. Instead, we propose a multi-stage and bidirectional fusion pipeline, which not only fully utilizes the characteristic of each modality, but maximizes the inter-modality complementarity as well.…”

PDC: Piecewise Depth Completion utilizing Superpixels

“…RIC-FLow does not use the raw matches directly but generate a superpixel flow from input matching to improve the efficiency of the model estimation. This concept was also adapted for scene flow estimation from sparse LiDAR and RGB image input [15]. Our approach, which we describe in detail in Section III, applies the superpixel method in the context of depth completion.…”