Residual 3D Scene Flow Learning with Context-Aware Feature Extraction

Aiming at obtaining structural information and 3D motion of dynamic scenes, scene flow estimation has been an interest of research in computer vision and computer graphics for a long time. It is also a fundamental task for various applications such as autonomous driving. Compared to previous methods that utilize image representations, many recent researches build upon the power of deep analysis and focus on point clouds representation to conduct 3D flow estimation. This paper comprehensively reviews the pioneering literature in scene flow estimation based on point clouds. Meanwhile, it delves into detail in learning paradigms and presents insightful comparisons between the state‐of‐the‐art methods using deep learning for scene flow estimation. Furthermore, this paper investigates various higher‐level scene understanding tasks, including object tracking, motion segmentation, etc. and concludes with an overview of foreseeable research trends for scene flow estimation.

show abstract

“…There are four main metrics to evaluate the predicted scene flow. More detailed equations of the following terms can be found in [WHWW21].…”

Section: Problem Statement and Taxonomymentioning

confidence: 99%

Deep Learning for Scene Flow Estimation on Point Clouds: A Survey and Prospective Trends

Xiang

Chen

et al. 2023

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…3D scene flow can be applied to object detection and tracking, [1][2][3] LiDAR odometry, [4] action recognition, [5] etc. Recently, some works [6][7][8][9][10][11] have been done to realize supervised estimation of 3D scene flow from two consecutive frames of point clouds. However, just like it is difficult to obtain the ground truth of optical flow, [12,13] the ground truth of 3D scene flow is also difficult to obtain.…”

Section: Introductionmentioning

confidence: 99%

SFGAN: Unsupervised Generative Adversarial Learning of 3D Scene Flow from the 3D Scene Self

Wang

Chaokang

Shen

et al. 2021

Advanced Intelligent Systems

Self Cite

View full text Add to dashboard Cite

Scene flow tracks the 3D motion of each point in adjacent point clouds. It provides fundamental 3D motion perception for autonomous driving and server robot. Although red green blue depth (RGBD) camera or light detection and ranging (LiDAR) capture discrete 3D points in space, the objects and motions usually are continuous in the macroworld. That is, the objects keep themselves consistent as they flow from the current frame to the next frame. Based on this insight, the generative adversarial networks (GAN) is utilized to self‐learn 3D scene flow without ground truth. The fake point cloud is synthesized from the predicted scene flow and the point cloud of the first frame. The adversarial training of the generator and discriminator is realized through synthesizing indistinguishable fake point cloud and discriminating the real point cloud and the synthesized fake point cloud. The experiments on Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI) dataset show that our method realizes promising results. Just as human, the proposed method can identify the similar local structures of two adjacent frames even without knowing the ground truth scene flow. Then, the local correspondence can be correctly estimated, and further the scene flow is correctly estimated. An interactive preprint version of the article can be found here: https://www.authorea.com/doi/full/10.22541/au.163335790.03073492.

show abstract

Residual 3D Scene Flow Learning with Context-Aware Feature Extraction

Cited by 2 publications

References 0 publications

Deep Learning for Scene Flow Estimation on Point Clouds: A Survey and Prospective Trends

Deep Learning for Scene Flow Estimation on Point Clouds: A Survey and Prospective Trends

SFGAN: Unsupervised Generative Adversarial Learning of 3D Scene Flow from the 3D Scene Self

Contact Info

Product

Resources

About