Panoster: End-to-end Panoptic Segmentation of LiDAR Point Clouds

Abstract

Panoptic segmentation has recently unified semantic and instance segmentation, previously addressed separately, thus taking a step further towards creating more comprehensive and efficient perception systems. In this paper, we present Panoster, a novel proposal-free panoptic segmentation method for point clouds. Unlike previous approaches relying on several steps to group pixels or points into objects, Panoster proposes a simplified framework incorporating a learning-based clustering solution to identify instances. At inference time, this acts as a class-agnostic semantic segmentation, allowing Panoster to be fast, while outperforming prior methods in terms of accuracy. Additionally, we showcase how our approach can be flexibly and effectively applied on diverse existing semantic architectures to deliver panoptic predictions.

EfficientLPS: Efficient LiDAR Panoptic Segmentation

“…Miliotto et al [6] and Gasperini et al [5] adopt the bottomup approach where instances are detected without region proposals. Miliotto et al use spherical projection of point clouds and predict offsets to the centroids for aiding clustering.…”

“…They also use 3D information available in the range images for trilinear upsampling in the decoder. Panoster [5] uses an instance head which directly provides the instance ids of the points from learnable clustering without any explicit grouping requirement. In addition to the spherical projectionbased method Rangenet++ [23] and Panoster [5] also show the implementation of their clustering mechanism using the point-based method KPConv [21] for semantic segmentation.…”

“…Panoster [5] uses an instance head which directly provides the instance ids of the points from learnable clustering without any explicit grouping requirement. In addition to the spherical projectionbased method Rangenet++ [23] and Panoster [5] also show the implementation of their clustering mechanism using the point-based method KPConv [21] for semantic segmentation. PanopticTrackNet [10] further unifies panoptic segmentation with multi-object tracking and provides temporally consistent instance labels.…”

“…Panoptic segmentation has been extensively studied in the image domain [1]- [4], facilitated by the ordered structure of images being supported by well-researched convolutional networks. However, only a handful of methods have been proposed for panoptic segmentation of LiDAR point clouds [5], [6]. LiDARs have become an indispensable sensor for autonomous vehicles due to their illumination independence and geometric description of the scene, making scene understanding using LiDAR point clouds an essential capability.…”

Unsupervised Domain Adaptation for LiDAR Panoptic Segmentation

“…Milioto et al [4] project the points onto a 2D image and predict their offsets to object centroids before grouping them in a post-processing step. Panoster [18] employs KPConv [19] together with a learnable clustering algorithm that removes the need for an additional post-processing stage to group points into instances. SMAC-Seg [20] uses a learnable multi-directional clustering along with a centroid-aware loss function to differentiate between object clusters.…”