Spatiotemporal Transformer Attention Network for 3D Voxel Level Joint Segmentation and Motion Prediction in Point Cloud

Wei, Zhensong; Qi, Xuewei; Bai, Zhengwei; Wu, Guoyuan; Nayak, Saswat; Peng, Hao; Barth, Matthew; Liu, Yongkang; Oguchi, Kimio

doi:10.1109/iv51971.2022.9827310

Cited by 2 publications

(4 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These methods represent the environment with BEV maps derived from point clouds and aim to predict the 2D displacement vector for each BEV cell along the horizontal plane. MotionNet (Wu, Chen, and Metaxas 2020) and BE-STI (Wei et al 2022) proposes to perform joint category perception and motion prediction from the BEV maps. LSTM-ED (Schreiber, Hoermann, and Dietmayer 2019) introduce convolutional LSTM (Shi et al 2015) to aggregate temporal context.…”

Section: Related Workmentioning

confidence: 99%

“…However, these approaches may face challenges when handling categories that have not been seen in the training set, mainly due to their reliance on object detection (Wu, Chen, and Metaxas 2020). To address this challenge, class-agnostic motion prediction task (Schreiber, Hoermann, and Dietmayer 2019;Wu, Chen, and Metaxas 2020;Wang et al 2022;Wei et al 2022) is proposed to provide complementary information. These methods take a sequence of previous point clouds as input and predict the future displacements for each Bird's Eye View (BEV) cell.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Semi-supervised Class-Agnostic Motion Prediction with Pseudo Label Regeneration and BEVMix

Wang,

Wu,

Pan

et al. 2024

AAAI

View full text Add to dashboard Cite

Class-agnostic motion prediction methods aim to comprehend motion within open-world scenarios, holding significance for autonomous driving systems. However, training a high-performance model in a fully-supervised manner always requires substantial amounts of manually annotated data, which can be both expensive and time-consuming to obtain. To address this challenge, our study explores the potential of semi-supervised learning (SSL) for class-agnostic motion prediction. Our SSL framework adopts a consistency-based self-training paradigm, enabling the model to learn from unlabeled data by generating pseudo labels through test-time inference. To improve the quality of pseudo labels, we propose a novel motion selection and re-generation module. This module effectively selects reliable pseudo labels and re-generates unreliable ones. Furthermore, we propose two data augmentation strategies: temporal sampling and BEVMix. These strategies facilitate consistency regularization in SSL. Experiments conducted on nuScenes demonstrate that our SSL method can surpass the self-supervised approach by a large margin by utilizing only a tiny fraction of labeled data. Furthermore, our method exhibits comparable performance to weakly and some fully supervised methods. These results highlight the ability of our method to strike a favorable balance between annotation costs and performance. Code will be available at https://github.com/kwwcv/SSMP.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Semi-supervised Class-Agnostic Motion Prediction with Pseudo Label Regeneration and BEVMix

Wang,

Wu,

Pan

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Key Technology of Automation Control Based on Artificial Intelligence Technology

Wang

2021

Advances in Intelligent Systems and Computing

View full text Add to dashboard Cite

Current state-of-the-art approaches for few-shot action recognition achieve promising performance by conducting frame-level matching on learned visual features. However, they generally suffer from two limitations: i) the matching procedure between local frames tends to be inaccurate due to the lack of guidance to force long-range temporal perception; ii) explicit motion learning is usually ignored, leading to partial information loss. To address these issues, we develop a Motion-augmented Long-short Contrastive Learning (MoLo) method that contains two crucial components, including a long-short contrastive objective and a motion autodecoder. Specifically, the long-short contrastive objective is to endow local frame features with long-form temporal awareness by maximizing their agreement with the global token of videos belonging to the same class. The motion autodecoder is a lightweight architecture to reconstruct pixel motions from the differential features, which explicitly embeds the network with motion dynamics. By this means, MoLo can simultaneously learn long-range temporal context and motion cues for comprehensive few-shot matching. To demonstrate the effectiveness, we evaluate MoLo on five standard benchmarks, and the results show that MoLo favorably outperforms recent advanced methods. The source code is available at https://github. com/alibaba-mmai-research/MoLo.

show abstract

Spatiotemporal Transformer Attention Network for 3D Voxel Level Joint Segmentation and Motion Prediction in Point Cloud

Cited by 2 publications

References 26 publications

Semi-supervised Class-Agnostic Motion Prediction with Pseudo Label Regeneration and BEVMix

Semi-supervised Class-Agnostic Motion Prediction with Pseudo Label Regeneration and BEVMix

Key Technology of Automation Control Based on Artificial Intelligence Technology

Contact Info

Product

Resources

About