4D Panoptic LiDAR Segmentation

Aygün, Mehmet; Ošep, Aljoša; Weber, Mark; Maximov, Maxim; Stachniss, Cyrill; Behley, Jens; Leal-Taixé, Laura

doi:10.1109/cvpr46437.2021.00548

Cited by 54 publications

(48 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…MOTS [18] proposed the multi-object tracking and segmentation task for the first time and MOTP [19] extended it to panoptic segmentation and 3D domain. 4D-PLS [3] explored tracking-by-segmentation paradigm, which can associate objects at point level. The core of 4D-PLS is the density-based clustering, which benefits from their semantic features.…”

Section: B 3d Multi-object Trackingmentioning

confidence: 99%

“…The training processes of these methods need the bounding box labels for the moving targets in each LiDAR frame. The 4D Panoptic LiDAR Segmentation (4D-PLS) [3] is based on a trackingby-segmentation paradigm and has shown remarkable performance in the LiDAR-based MOT task. However, the labeling of the segmentation task is more expensive, even with many advanced tools [4].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learning Moving-Object Tracking with FMCW LiDAR

Gu¹,

Cheng²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

In this paper, we propose a learning-based moving-object tracking method utilizing our newly developed LiDAR sensor, Frequency Modulated Continuous Wave (FMCW) LiDAR. Compared with most existing commercial LiDAR sensors, our FMCW LiDAR can provide additional Doppler velocity information to each 3D point of the point clouds. Benefiting from this, we can generate instance labels as ground truth in a semi-automatic manner. Given the labels, we propose a contrastive learning framework, which pulls together the features from the same instance in embedding space and pushes apart the features from different instances, to improve the tracking quality. Extensive experiments are conducted on our recorded driving data, and the results show that our method outperforms the baseline methods by a large margin.

show abstract

Section: B 3d Multi-object Trackingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Learning Moving-Object Tracking with FMCW LiDAR

Gu¹,

Cheng²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…However, KITTI does not provide dense (per-point) 3D labels on the point cloud. Closely related to our work, ApolloScape [43] annotates static scene elements in the 3D space 4 and projects them to the 2D image space, followed by manual annotation of dynamic objects in images. In this work, we annotate both static and dynamic objects in 3D, providing coherent annotations for dynamic objects both in 2D and 3D.…”

Section: Datasetsmentioning

confidence: 99%

“…For instance, Cityscapes [24] offers a benchmark suite for pixel and instance-level semantic segmentation as well as 3D vehicle detection. SemanticKITTI [4], [7] hosts lidar segmentation challenges to predict the category of every point. For datasets including both 2D and 3D annotations, KITTI [31], nuScenes [17], and ApolloScape [43] provide benchmarks on a set of vision tasks including detection, stereo, localization, multi-object tracking, and segmentation in both 2D and 3D, etc.…”

Section: Datasetsmentioning

confidence: 99%

See 1 more Smart Citation

KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D

Liao¹,

Xie²,

Geiger³

2021

Preprint

View full text Add to dashboard Cite

For the last few decades, several major subfields of artificial intelligence including computer vision, graphics, and robotics have progressed largely independently from each other. Recently, however, the community has realized that progress towards robust intelligent systems such as self-driving cars requires a concerted effort across the different fields. This motivated us to develop KITTI-360, successor of the popular KITTI dataset. KITTI-360 is a suburban driving dataset which comprises richer input modalities, comprehensive semantic instance annotations and accurate localization to facilitate research at the intersection of vision, graphics and robotics. For efficient annotation, we created a tool to label 3D scenes with bounding primitives and developed a model that transfers this information into the 2D image domain, resulting in over 150k semantic and instance annotated images and 1B annotated 3D points. Moreover, we established benchmarks and baselines for several tasks relevant to mobile perception, encompassing problems from computer vision, graphics, and robotics on the same dataset. KITTI-360 will enable progress at the intersection of these research areas and thus contributing towards solving one of our grand challenges: the development of fully autonomous self-driving systems.

show abstract