4D-StOP: Panoptic Segmentation of 4D LiDAR Using Spatio-Temporal Object Proposal Generation and Aggregation

Kreuzberg, Lars; Zulfikar, Idil Esen; Mahadevan, Sabarinath; Engelmann, Francis; Leibe, Bastian

doi:10.1007/978-3-031-25056-9_34

Cited by 19 publications

(8 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It accumulates point clouds in sequential timestamps to a common frame and applies segmentation on the aggregated point clouds. 4D-DS-Net [18] and 4D-StOP [21] also followed this pipeline but used different clustering strategies. 4D-PLS predicted the closeness to the instance centers for each point.…”

Section: D Panoptic Segmentationmentioning

confidence: 99%

“…4D-PLS predicted the closeness to the instance centers for each point. 4D-DS-Net [18] and 4D-StOP [21] predicted the offset to the instance center from each point. CA-Net [25] directly used an off-the-shelf 3D panoptic segmentation network and learned the temporal instances association through contrastive learning.…”

Section: D Panoptic Segmentationmentioning

confidence: 99%

“…We describe necessary adaptations to E2PN in Sec. 4.2, which allow us to build equivariant models on top of SOTA 4D panoptic segmentation network 4D-PLS [1] and 4D-StOP [21], both of which are based on KPConv. We refer to our equivariant models as Eq-4D-PLS and Eq-4D-StOP, respectively.…”

Section: Network Overviewmentioning

confidence: 99%

“…Our baseline models 4D-PLS [1] and 4D-StOP [21] have different prediction heads. Their semantic segmentation heads are similar, but they employ different clustering approaches for instance segmentation.…”

Section: Equivariant Prediction Headsmentioning

confidence: 99%

“…While there are a few existing methods to solve 4D panoptic segmentation [1,21], they ignore the rich, inherent symmetries present in this task. For example, for the point cloud of an object instance, its center is invariant to rotations, and the offset vector from any point on the object to the center is attached to and thus rotates along with the body frame.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

4D Panoptic Segmentation as Invariant and Equivariant Field Prediction

Zhu¹,

Han²,

Chen³

et al. 2023

Preprint

View full text Add to dashboard Cite

In this paper, we develop rotation-equivariant neural networks for 4D panoptic segmentation. 4D panoptic segmentation is a recently established benchmark task for autonomous driving, which requires recognizing semantic classes and object instances on the road based on LiDAR scans, as well as assigning temporally consistent IDs to instances across time. We observe that the driving scenario is symmetric to rotations on the ground plane. Therefore, rotation-equivariance could provide better generalization and more robust feature learning. Specifically, we review the object instance clustering strategies, and restate the centerness-based approach and the offset-based approach as the prediction of invariant scalar fields and equivariant vector fields. Other sub-tasks are also unified from this perspective, and different invariant and equivariant layers are designed to facilitate their predictions. Through evaluation on the standard 4D panoptic segmentation benchmark of SemanticKITTI, we show that our equivariant models achieve higher accuracy with lower computational costs compared to their non-equivariant counterparts. Moreover, our method sets the new state-of-the-art performance and achieves 1st place on the SemanticKITTI 4D Panoptic Segmentation leaderboard.

show abstract

Section: D Panoptic Segmentationmentioning

confidence: 99%