An End-To-End Network for Panoptic Segmentation

Liu, Huanyu; Peng, Chao; Yu, Changqian; Wang, Jingbo; Xu, Lu; Yu, Gang; Jiang, Wei

doi:10.1109/cvpr.2019.00633

Cited by 156 publications

(97 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The work in [14] attempts to introduce a unified architecture related to our ideas, however, the reported results remain significantly below those of reported state-of-the-art methods. Independently and simultaneously to our paper, a number of works [23,29,35,54,55] have proposed panoptic segmentation provided by a single deep network, confirming the importance of this task to the field. While comparable in complexity and architecture, we obtain improved performance on challenging street-level image datasets like Cityscapes and Mapillary Vistas.…”

Section: Related Worksupporting

confidence: 77%

Seamless Scene Segmentation

Porzi¹,

Bulò²,

Colovic³

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

234

203

View full text Add to dashboard Cite

In this work we introduce a novel, CNN-based architecture that can be trained end-to-end to deliver seamless scene segmentation results. Our goal is to predict consistent semantic segmentation and detection results by means of a panoptic output format, going beyond the simple combination of independently trained segmentation and detection models. The proposed architecture takes advantage of a novel segmentation head that seamlessly integrates multi-scale features generated by a Feature Pyramid Network with contextual information conveyed by a lightweight DeepLab-like module. As additional contribution we review the panoptic metric and propose an alternative that overcomes its limitations when evaluating non-instance categories. Our proposed network architecture yields state-ofthe-art results on three challenging street-level datasets, i.e. Cityscapes, Indian Driving Dataset and Mapillary Vistas.

show abstract

Section: Related Worksupporting

confidence: 77%

Seamless Scene Segmentation

Porzi¹,

Bulò²,

Colovic³

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

234

203

View full text Add to dashboard Cite

show abstract

“…Finally we note that the panoptic segmentation task was featured as a challenge track by both the COCO [25] and Mapillary Vistas [35] recognition challenges and that the proposed task has already begun to gain traction in the community (e.g. [23,48,49,27,22,21,17] address PS).…”

Section: Future Of Panoptic Segmentationmentioning

confidence: 99%

Panoptic Segmentation

Kirillov

Girshick

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

1,326

1,059

View full text Add to dashboard Cite

We propose and study a task we name panoptic segmentation (PS). Panoptic segmentation unifies the typically distinct tasks of semantic segmentation (assign a class label to each pixel) and instance segmentation (detect and segment each object instance). The proposed task requires generating a coherent scene segmentation that is rich and complete, an important step toward real-world vision systems. While early work in computer vision addressed related image/scene parsing tasks, these are not currently popular, possibly due to lack of appropriate metrics or associated recognition challenges. To address this, we propose a novel panoptic quality (PQ) metric that captures performance for all classes (stuff and things) in an interpretable and unified manner. Using the proposed metric, we perform a rigorous study of both human and machine performance for PS on three existing datasets, revealing interesting insights about the task. The aim of our work is to revive the interest of the community in a more unified view of image segmentation.

show abstract

“…Furthermore, we also establish the bond between proposal-based instance and FCN based semantic segmentation. Most recently works include [22,29,40]. Instance Segmentation: Instance segmentation aims at discriminating different instances of the same object.…”

Section: Related Workmentioning

confidence: 99%

Attention-Guided Unified Network for Panoptic Segmentation

Chen

Zhu

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

271

186

View full text Add to dashboard Cite

This paper studies panoptic segmentation, a recently proposed task which segments foreground (FG) objects at the instance level as well as background (BG) contents at the semantic level. Existing methods mostly dealt with these two problems separately, but in this paper, we reveal the underlying relationship between them, in particular, FG objects provide complementary cues to assist BG understanding. Our approach, named the Attention-guided Unified Network (AUNet), is a unified framework with two branches for FG and BG segmentation simultaneously. Two sources of attentions are added to the BG branch, namely, RPN and FG segmentation mask to provide object-level and pixellevel attentions, respectively. Our approach is generalized to different backbones with consistent accuracy gain in both FG and BG segmentation, and also sets new state-of-thearts both in the MS-COCO (46.5% PQ) and Cityscapes (59.0% PQ) benchmarks.

show abstract

An End-To-End Network for Panoptic Segmentation

Cited by 156 publications

References 53 publications

Seamless Scene Segmentation

Seamless Scene Segmentation

Panoptic Segmentation

Attention-Guided Unified Network for Panoptic Segmentation

Contact Info

Product

Resources

About