Consistency Guided Scene Flow Estimation

Chen, Yuhua; Gool, Luc Van; Schmid, Cordelia; Sminchisescu, Cristian

doi:10.1007/978-3-030-58571-6_8

Cited by 6 publications

(4 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Scene Flow from Stereo and RGB-D Videos. Many methods [4,13,19,20,42,46,49] are proposed to estimate scene flow from stereo videos or RGB-D frames. These works formulate scene flow estimation as a problem jointly estimating disparity map and optical flow from stereo frames [32].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Learning Scene Flow in 3D Point Clouds with Noisy Pseudo Labels

Li¹,

Cheng²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

We propose a novel scene flow method that captures 3D motions from point clouds without relying on ground-truth scene flow annotations. Due to the irregularity and sparsity of point clouds, it is expensive and time-consuming to acquire ground-truth scene flow annotations. Some state-of-the-art approaches train scene flow networks in a self-supervised learning manner via approximating pseudo scene flow labels from point clouds. However, these methods fail to achieve the performance level of fully supervised methods, due to the limitations of point cloud such as sparsity and lacking color information. To provide an alternative, we propose a novel approach that utilizes monocular RGB images and point clouds to generate pseudo scene flow labels for training scene flow networks. Our pseudo label generation module infers pseudo scene labels for point clouds by jointly leveraging rich appearance information in monocular images and geometric information of point clouds. To further reduce the negative effect of noisy pseudo labels on the training, we propose a noisy-labelaware training scheme by exploiting the geometric relations of points. Experiment results show that our method not only outperforms state-of-the-art self-supervised approaches but also outperforms some supervised approaches that use accurate ground-truth flows.

show abstract

Section: Related Workmentioning

confidence: 99%

“…These works formulate scene flow estimation as a problem jointly estimating disparity map and optical flow from stereo frames [32]. Recent works [4,19,27] propose neural-networkbased models to estimate scene flow from stereo video. Some works [37,39] focus on estimating scene flow from RGB-D videos.…”

Section: Related Workmentioning

confidence: 99%

Learning Scene Flow in 3D Point Clouds with Noisy Pseudo Labels

Li¹,

Cheng²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Scene Flow from Stereo and RGB-D Videos. Estimating scene flow from stereo videos have been studied for years [6,56,59,22,24,50]. Many works estimate scene flow by jointly estimating stereo matching and optical flow from consecutive stereo frames [37].…”

Section: Contributions Our Contributions Are Fourfoldmentioning

confidence: 99%

“…Similar to optical flow, traditional methods formulate scene flow estimation as an energy minimization problem [18,59]. Recent works estimate scene flow from stereo video using neural networks [6]. For example, networks for disparity estimation and optical flow are combined in [22,32].…”

Section: Contributions Our Contributions Are Fourfoldmentioning

confidence: 99%

SCTN: Sparse Convolution-Transformer Network for Scene Flow Estimation

Li¹,

Cheng²,

Giancola³

et al. 2021

Preprint

View full text Add to dashboard Cite

We propose a novel scene flow estimation approach to capture and infer 3D motions from point clouds. Estimating 3D motions for point clouds is challenging, since a point cloud is unordered and its density is significantly non-uniform. Such unstructured data poses difficulties in matching corresponding points between point clouds, leading to inaccurate flow estimation. We propose a novel architecture named Sparse Convolution-Transformer Network (SCTN) that equips the sparse convolution with the transformer. Specifically, by leveraging the sparse convolution, SCTN transfers irregular point cloud into locally consistent flow features for estimating continuous and consistent motions within an object/local object part. We further propose to explicitly learn point relations using a point transformer module, different from exiting methods. We show that the learned relation-based contextual information is rich and helpful for matching corresponding points, benefiting scene flow estimation. In addition, a novel loss function is proposed to adaptively encourage flow consistency according to feature similarity. Extensive experiments demonstrate that our proposed approach achieves a new state of the art in scene flow estimation. Our approach achieves an error of 0.038 and 0.037 (EPE3D) on FlyingThings3D and KITTI Scene Flow respectively, which significantly outperforms previous methods by large margins.

show abstract