FlowNet3D++: Geometric Losses For Deep Scene Flow Estimation

Wang, Zirui; Li, Shuda; Howard-Jenkins, Henry; Prisacariu, Victor Adrian; Chen, Min

doi:10.1109/wacv45572.2020.9093302

Cited by 119 publications

(73 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There is a rich literature of building learned models for scene flow using end-to-end learned architectures [30], [31], [8], [7], [14], [13], [32], [25] as well as hybrid architectures [33], [34], [35]. We discuss these in Section V in conjunction with building a scalable baseline model that operates in real time.…”

Section: Models For Learning Scene Flowmentioning

confidence: 99%

“…Furthermore, previous datasets cover a smaller area, e.g., the KITTI scene flow dataset covers 1/5th the area of our proposed dataest. This allows for different subsampling tradeoffs and inspired a class of models that are not able to tractably scale training and inference beyond ∼10K points [7], [8], [9], [13], [14], making the usage of such models impractical in real world AV scenes which often contain 100K -1000K points.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Scalable Scene Flow From Point Clouds in the Real World

Jund

Sweeney²,

Abdo³

et al. 2022

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Autonomous vehicles operate in highly dynamic environments necessitating an accurate assessment of which aspects of a scene are moving and where they are moving to. A popular approach to 3D motion estimation, termed scene flow, is to employ 3D point cloud data from consecutive LiDAR scans, although such approaches have been limited by the small size of real-world, annotated LiDAR data. In this work, we introduce a new large-scale dataset for scene flow estimation derived from corresponding tracked 3D objects, which is ∼1,000× larger than previous real-world datasets in terms of the number of annotated frames. We demonstrate how previous works were bounded by the amount of real LiDAR data available, suggesting that larger datasets are required to achieve state-of-the-art predictive performance. Furthermore, we show how previous heuristics such as down-sampling heavily degrade performance, motivating a new class of models that are tractable on the full point cloud.To address this issue, we introduce the FastFlow3D architecture which provides real time inference on the full point cloud. Additionally, we design human-interpretable metrics that better capture real world aspects by accounting for ego-motion and providing breakdowns per object type. We hope that this dataset may provide new opportunities for developing real world scene flow systems.

show abstract

Section: Models For Learning Scene Flowmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Scalable Scene Flow From Point Clouds in the Real World

Jund

Sweeney²,

Abdo³

et al. 2022

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

show abstract

“…Un-/self-supervised approaches [31,32,69] aim to overcome the dependency on accurate, diverse labeled data, which is not easy to obtain. Approaches using sequences of RGB-D images [16,19,21,37,50,51] or 3D point clouds [4,17,35,45,70,72] have been also proposed, exploiting an already given 3D sparse point input.…”

Section: Related Workmentioning

confidence: 99%

Self-Supervised Monocular Scene Flow Estimation

Hur

Roth

2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

113

View full text Add to dashboard Cite

Estimating 3D scene flow from a sequence of monocular images has been gaining increased attention due to the simple, economical capture setup. Owing to the severe illposedness of the problem, the accuracy of current methods has been limited, especially that of efficient, real-time approaches. In this paper, we introduce a multi-frame monocular scene flow network based on self-supervised learning, improving the accuracy over previous networks while retaining real-time efficiency. Based on an advanced twoframe baseline with a split-decoder design, we propose (i) a multi-frame model using a triple frame input and convolutional LSTM connections, (ii) an occlusion-aware census loss for better accuracy, and (iii) a gradient detaching strategy to improve training stability. On the KITTI dataset, we observe state-of-the-art accuracy among monocular scene flow methods based on self-supervised learning.

show abstract

“…As suggested in [19] for the task of scene flow estimation we investigate regularization by adding the cosine distance Lcos dist between the ground truth y = [t, r] T and predictionŷ = [t,r] T , according to eq. (5), as a regularization term to the overall loss function.…”

Section: Regularizationmentioning

confidence: 99%

D3dlo: Deep 3d Lidar Odometry

Adis

Horst

Wien

2021

2021 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

LiDAR odometry (LO) describes the task of finding an alignment of subsequent LiDAR point clouds. This alignment can be used to estimate the motion of the platform where the LiDAR sensor is mounted on. Currently, on the well-known KITTI Vision Benchmark Suite state-of-the-art algorithms are non-learning approaches. We propose a network architecture that learns LO by directly processing 3D point clouds. It is trained on the KITTI dataset in an end-to-end manner without the necessity of pre-defining corresponding pairs of points. An evaluation on the KITTI Vision Benchmark Suite shows similar performance to a previously published work, DeepCLR [1], even though our model uses only around 3.56% of the number of network parameters thereof. Furthermore, a plane point extraction is applied which leads to a marginal performance decrease while simultaneously reducing the input size by up to 50%.

show abstract

FlowNet3D++: Geometric Losses For Deep Scene Flow Estimation

Cited by 119 publications

References 37 publications

Scalable Scene Flow From Point Clouds in the Real World

Scalable Scene Flow From Point Clouds in the Real World

Self-Supervised Monocular Scene Flow Estimation

D3dlo: Deep 3d Lidar Odometry

Contact Info

Product

Resources

About