Fast Panoptic Segmentation Network

Abstract

In this work, we present an end-to-end network for fast panoptic segmentation. This network, called Fast Panoptic Segmentation Network (FPSNet), does not require computationally costly instance mask predictions or merging heuristics. This is achieved by casting the panoptic task into a custom dense pixel-wise classification task, which assigns a class label or an instance id to each pixel. We evaluate FPSNet on the Cityscapes and Pascal VOC datasets, and find that FPSNet is faster than existing panoptic segmentation methods, while achieving better or similar panoptic segmentation performance. On the Cityscapes validation set, we achieve a Panoptic Quality score of 55.1%, at prediction times of 114 milliseconds for images with a resolution of 1024x2048 pixels. For lower resolutions of the Cityscapes dataset and for the Pascal VOC dataset, FPSNet runs at 22 and 35 frames per second, respectively.

Fast Panoptic Segmentation with Soft Attention Embeddings

“…Lately, oneshot detectors, such as RetinaNet [17] or FCOS [13], have achieved great progress and even surpassed two-stage detectors on public benchmarks. Motivated by their benefits, FPSNet [10] builds upon RetinaNet [17] and achieves a fast inference speed. DenseBox [11] leverages the idea of reusing discarded dense object proposals by matching them with final object bounding boxes.…”

“…Most recent approaches [6,7] focus on achieving high accuracy on public benchmarks by designing novel semantic segmentation heads on top of Mask R-CNN [8] instance segmentation network or by designing clever merging heuristics within the network [9]. However, significant progress has been achieved by fast networks, starting from [10], which builds upon a one-stage detector, to [11], where dense bounding boxes are clustered into instance masks using semantic segmentation and finally, the proposal-free method [12], which predicts instance centers and regresses instance center offsets.…”

Automotive Intelligence Embedded in Electric Connected Autonomous and Shared Vehicles Technology for Sustainable Green Mobility

“…The ultimate objective for the entire system is to take measurements from agent's sensors, such as cameras, LiDARs, or RaDARs, and estimate a world model, its perception system. As is typical in 3D environment sensing, designing a comprehensive "hand-crafted" model is not feasible; instead, the model is learned from training data, using machine learning, such as semantic segmentation (Meletis and Dubbelman, 2018), part-level panoptic segmentation (de Geus et al, 2020), human pose estimation (Cao et al, 2021), action recognition (Li Z. et al, 2018), and depth and flow (Zou et al, 2018).…”

YOLO-based Panoptic Segmentation Network

“…And as can be seen in table III, using a smaller images increases inference speed by 2× at the expense of 3 Panoptic Quality points. PQ, SQ and RQ correspond to the Panoptic Quality, Architecture PQ(%) Time(ms) JSISNet [17] 17.6 n/a AUNet [5] 59.0 n/a Panoptic FPN [3] 58.1 n/a Single Network PS [18] 42.9 590 DeeperLab [8] 56.53 308 Panoptic Deeplab [9] 63.0 175 AdapIS [10] 62.0 n/a FPSNet [19] 55.1 114 Real-Time PS [20] 58…”