FH-Net: A Fast Hierarchical Network for Scene Flow Estimation on Real-World Point Clouds

Ding, Lihe; Dong, Shaocong; Xu, Tingfa; Xu, Xinli; Wang, Jie; Li, Jianan

doi:10.1007/978-3-031-19842-7_13

Cited by 11 publications

(10 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…FH‐Net [DDX*22] deals with multi‐scale flows from different layers with a much faster speed. To this end, FH‐Net extracts keypoint features via hierarchical Trans‐flow layer.…”

Section: Methodsmentioning

confidence: 99%

Deep Learning for Scene Flow Estimation on Point Clouds: A Survey and Prospective Trends

Xiang

Chen

et al. 2023

Computer Graphics Forum

View full text Add to dashboard Cite

Aiming at obtaining structural information and 3D motion of dynamic scenes, scene flow estimation has been an interest of research in computer vision and computer graphics for a long time. It is also a fundamental task for various applications such as autonomous driving. Compared to previous methods that utilize image representations, many recent researches build upon the power of deep analysis and focus on point clouds representation to conduct 3D flow estimation. This paper comprehensively reviews the pioneering literature in scene flow estimation based on point clouds. Meanwhile, it delves into detail in learning paradigms and presents insightful comparisons between the state‐of‐the‐art methods using deep learning for scene flow estimation. Furthermore, this paper investigates various higher‐level scene understanding tasks, including object tracking, motion segmentation, etc. and concludes with an overview of foreseeable research trends for scene flow estimation.

show abstract

“…FH‐Net [DDX*22] deals with multi‐scale flows from different layers with a much faster speed. To this end, FH‐Net extracts keypoint features via hierarchical Trans‐flow layer.…”

Section: Methodsmentioning

confidence: 99%

Deep Learning for Scene Flow Estimation on Point Clouds: A Survey and Prospective Trends

Xiang

Chen

et al. 2023

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…Recently, deep learning has demonstrated powerful capabilities in end-to-end learning of scene flow estimation from stereo inputs [24,32,41]. Additionally, approaches that leverage pre-existing 3D structure through inputs of RGB-D sequences [31,39,45,33] or Lidar points [28,56,38,55,12,11,52] have also been proposed for various scenarios. Monocular scene flow.…”

Section: Related Workmentioning

confidence: 99%

“…based on various input modalities, including stereo images [3,24,32,41,51,40], RGB-D pairs [31,39,45,33], or Lidar points [28,18,54,56,38,55,12,7,11,52]. These methods, however, either require strict sensor calibrations (e.g., stereo-based), or expensive devices (e.g., RGB-D or Lidar-based) for achieving satisfactory performance, which restricts their widespread applications.…”

Section: Introductionmentioning

confidence: 99%

Self-Supervised Ego-Motion Estimation Based on Multi-Layer Fusion of RGB and Inferred Depth

Jiang

Taira

Miyashita

et al. 2022

2022 International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

Self-supervised monocular scene flow estimation, aiming to understand both 3D structures and 3D motions from two temporally consecutive monocular images, has received increasing attention for its simple and economical sensor setup. However, the accuracy of current methods suffers from the bottleneck of less-efficient network architecture and lack of motion rigidity for regularization. In this paper, we propose a superior model named EMR-MSF by borrowing the advantages of network architecture design under the scope of supervised learning. We further impose explicit and robust geometric constraints with an elaborately constructed ego-motion aggregation module where a rigidity soft mask is proposed to filter out dynamic regions for stable ego-motion estimation using static regions. Moreover, we propose a motion consistency loss along with a mask regularization loss to fully exploit static regions. Several efficient training strategies are integrated including a gradient detachment technique and an enhanced view synthesis process for better performance. Our proposed method outperforms the previous self-supervised works by a large margin and catches up to the performance of supervised methods. On the KITTI scene flow benchmark, our approach improves the SF-all metric of the state-of-theart self-supervised monocular method by 44% and demonstrates superior performance across sub-tasks including depth and visual odometry, amongst other self-supervised single-task or multi-task methods.

show abstract

“…The development of 3D sensors has led to a growing interest in scene flow estimation from point clouds. Various approaches [8], [9], [10], [13], [14], [15], [45], [46], [47], [48], [49], [50], [51], [52], [53] have been proposed to achieve scene flow estimation in a fully supervised manner. Specifically, 3DFlow [51] estimates scene flow by establishing all-to-all flow embedding.…”

Section: Related Workmentioning

confidence: 99%

RigidFlow: Self-Supervised Scene Flow Learning on Point Clouds by Local Rigidity Prior

Zhang

Lin

et al. 2022

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

In this article, we investigate self-supervised 3D scene flow estimation and class-agnostic motion prediction on point clouds. A realistic scene can be well modeled as a collection of rigidly moving parts, therefore its scene flow can be represented as a combination of the rigid motion of these individual parts. Building upon this observation, we propose to generate pseudo scene flow labels for self-supervised learning through piecewise rigid motion estimation, in which the source point cloud is decomposed into local regions and each region is treated as rigid. By rigidly aligning each region with its potential counterpart in the target point cloud, we obtain a region-specific rigid transformation to generate its pseudo flow labels. To mitigate the impact of potential outliers on label generation, when solving the rigid registration for each region, we alternately perform three steps: establishing point correspondences, measuring the confidence for the correspondences, and updating the rigid transformation based on the correspondences and their confidence. As a result, confident correspondences will dominate label generation and a validity mask will be derived for the generated pseudo labels. By using the pseudo labels together with their validity mask for supervision, models can be trained in a self-supervised manner. Extensive experiments on FlyingThings3D and KITTI datasets demonstrate that our method achieves new state-of-the-art performance in self-supervised scene flow learning, without any ground truth scene flow for supervision, even performing better than some supervised counterparts. Additionally, our method is further extended to class-agnostic motion prediction and significantly outperforms previous state-of-the-art self-supervised methods on nuScenes dataset.

show abstract

FH-Net: A Fast Hierarchical Network for Scene Flow Estimation on Real-World Point Clouds

Cited by 11 publications

References 42 publications

Deep Learning for Scene Flow Estimation on Point Clouds: A Survey and Prospective Trends

Deep Learning for Scene Flow Estimation on Point Clouds: A Survey and Prospective Trends

Self-Supervised Ego-Motion Estimation Based on Multi-Layer Fusion of RGB and Inferred Depth

RigidFlow: Self-Supervised Scene Flow Learning on Point Clouds by Local Rigidity Prior

Contact Info

Product

Resources

About