SLIM: Self-Supervised LiDAR Scene Flow and Motion Segmentation

Baur, Stefan A.; Emmerichs, David; Moosmann, Frank; Pinggera, Peter; Ommer, Björn; Geiger, Andreas

doi:10.1109/iccv48922.2021.01288

Cited by 41 publications

(36 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…SLIM [BEM*21] removes the annotation requirement constraint on realistic data by integrating the self‐supervised scene flow estimation and the motion segmentation framework. SLIM presents that the motion segmentation signal can be generated by detecting the discrepancy between raw flow predictions and rigid ego‐motion.…”

Section: Methodsmentioning

confidence: 99%

“…From pedestrians walking at a constant speed to high‐speed vehicles, the issue of detecting objects of interest can be addressed by segmenting the underlying motions. Intuitively, the segmentation of different motion fields is conducted through classifying the point cloud into moving bodies and stationary backgrounds [BEM*21]. Discontinuities in the scene flow are key cues for segmenting a point cloud into several individual objects with different motion fields.…”

Section: Applicationsmentioning

confidence: 99%

“…Discontinuities in the scene flow are key cues for segmenting a point cloud into several individual objects with different motion fields. Recently, SLIM [BEM*21] proposes a self‐supervised learning approach and presents motion segmentation results that illustrate the effectiveness of jointly estimating scene flow and segmenting motion fields. The performance of SLIM is improved with self‐supervised motion segmentation signal as it achieves a mIoU score of 59.5% with a sensitivity of 73.1% on KITTI dataset.…”

Section: Applicationsmentioning

confidence: 99%

See 2 more Smart Citations

Deep Learning for Scene Flow Estimation on Point Clouds: A Survey and Prospective Trends

Xiang

Chen

et al. 2023

Computer Graphics Forum

View full text Add to dashboard Cite

Aiming at obtaining structural information and 3D motion of dynamic scenes, scene flow estimation has been an interest of research in computer vision and computer graphics for a long time. It is also a fundamental task for various applications such as autonomous driving. Compared to previous methods that utilize image representations, many recent researches build upon the power of deep analysis and focus on point clouds representation to conduct 3D flow estimation. This paper comprehensively reviews the pioneering literature in scene flow estimation based on point clouds. Meanwhile, it delves into detail in learning paradigms and presents insightful comparisons between the state‐of‐the‐art methods using deep learning for scene flow estimation. Furthermore, this paper investigates various higher‐level scene understanding tasks, including object tracking, motion segmentation, etc. and concludes with an overview of foreseeable research trends for scene flow estimation.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Applicationsmentioning

confidence: 99%

Section: Applicationsmentioning

confidence: 99%

See 1 more Smart Citation

Deep Learning for Scene Flow Estimation on Point Clouds: A Survey and Prospective Trends

Xiang

Chen

et al. 2023

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…Scene flow estimation [1] aims to generate a 3D motion field of a dynamic scene. As a fundamental representation of dynamics, scene flow can be applied in various tasks, such as motion segmentation [2], 3d object detection [3], and point cloud accumulation [4], as well as multiple downstream applications including robotics and autonomous driving [5], [6]. In recent years, with the widespread application of 3D sensors and the rise of deep learning techniques for point cloud processing, learning scene flow directly from 3D point clouds has attracted increasing research attention.…”

Section: Introductionmentioning

confidence: 99%

“…the self-supervised setting, in most previous approaches [2], [11], [12], [17], [18], [19], [20], [21], [22], [23], models estimate scene flow between two point clouds, and then the estimated scene flow is used to warp the source point cloud to match the target one. The main supervision signal is obtained by minimizing the discrepancy between the warped point cloud and the target point cloud, that is, by minimizing the distance between corresponding points in the two point clouds.…”

Section: Introductionmentioning

confidence: 99%

RigidFlow: Self-Supervised Scene Flow Learning on Point Clouds by Local Rigidity Prior

Zhang

Lin

et al. 2022

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

In this article, we investigate self-supervised 3D scene flow estimation and class-agnostic motion prediction on point clouds. A realistic scene can be well modeled as a collection of rigidly moving parts, therefore its scene flow can be represented as a combination of the rigid motion of these individual parts. Building upon this observation, we propose to generate pseudo scene flow labels for self-supervised learning through piecewise rigid motion estimation, in which the source point cloud is decomposed into local regions and each region is treated as rigid. By rigidly aligning each region with its potential counterpart in the target point cloud, we obtain a region-specific rigid transformation to generate its pseudo flow labels. To mitigate the impact of potential outliers on label generation, when solving the rigid registration for each region, we alternately perform three steps: establishing point correspondences, measuring the confidence for the correspondences, and updating the rigid transformation based on the correspondences and their confidence. As a result, confident correspondences will dominate label generation and a validity mask will be derived for the generated pseudo labels. By using the pseudo labels together with their validity mask for supervision, models can be trained in a self-supervised manner. Extensive experiments on FlyingThings3D and KITTI datasets demonstrate that our method achieves new state-of-the-art performance in self-supervised scene flow learning, without any ground truth scene flow for supervision, even performing better than some supervised counterparts. Additionally, our method is further extended to class-agnostic motion prediction and significantly outperforms previous state-of-the-art self-supervised methods on nuScenes dataset.

show abstract