Multi-instance object segmentation with occlusion handling

Chen, Yi-Ting; Liu, Xiaokai; Yang, Ming–Hsuan

doi:10.1109/cvpr.2015.7298969

Cited by 84 publications

(73 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some of the research that was capable of handling occlusions [4,11,12,36] also applied object completion to occluded objects. Those methods, however, have been shown to work only on specific object categories and relied on available shape models or depth inputs.…”

Section: Object Completion and Inpaintingmentioning

confidence: 99%

“…The concept of amodal segmentation has just emerged in the last few years [10,21,39], though similar problems had actually been addressed years before in many applications including detection [11,12,19], segmentation [4,36], reconstruction [13,31] and so on. Traditional approaches, however have usually relied on depth information or focused on specific object categories, while recent methods have solely used RGB images only and targeted objects of arbitrary categories.…”

Section: Amodal Segmentationmentioning

confidence: 99%

See 1 more Smart Citation

Learning Semantics-aware Distance Map with Semantics Layering Network for Amodal Instance Segmentation

Zhang

Chen

Xie

et al. 2019

Proceedings of the 27th ACM International Conference on Multimedia

View full text Add to dashboard Cite

In this work, we demonstrate yet another approach to tackle the amodal segmentation problem. Specifically, we first introduce a new representation, namely a semantics-aware distance map (sem-dist map), to serve as our target for amodal segmentation instead of the commonly used masks and heatmaps. The sem-dist map is a kind of level-set representation, of which the different regions of an object are placed into different levels on the map according to their visibility. It is a natural extension of masks and heatmaps, where modal, amodal segmentation, as well as depth order information, are all well-described. Then we also introduce a novel convolutional neural network (CNN) architecture, which we refer to as semantic layering network, to estimate sem-dist maps layer by layer, from the global-level to the instance-level, for all objects in an image. Extensive experiments on the COCOA and D2SA datasets have demonstrated that our framework can predict amodal segmentation, occlusion and depth order with state-of-the-art performance.

show abstract

Section: Object Completion and Inpaintingmentioning

confidence: 99%

Section: Amodal Segmentationmentioning

confidence: 99%

Learning Semantics-aware Distance Map with Semantics Layering Network for Amodal Instance Segmentation

Zhang

Chen

Xie

et al. 2019

Proceedings of the 27th ACM International Conference on Multimedia

View full text Add to dashboard Cite

show abstract

“…Most detection-based approaches use boxes as the intermediate representation for objects (see middle column) which do not contain information of object pose and shape. On the contrary, shapes are more informative (see right column) and have been used by numerous algorithms to help object segmentation [1,46,19,6,45]. As the pixels of novel objects may appear very different, we hypothesize that shapes could be leveraged to improve generalization as well.…”

Section: Introductionmentioning

confidence: 99%

ShapeMask: Learning to Segment Novel Objects by Refining Shape Priors

Kuo

Angelova

Malik

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

131

View full text Add to dashboard Cite

Instance segmentation aims to detect and segment individual objects in a scene. Most existing methods rely on precise mask annotations of every category. However, it is difficult and costly to segment objects in novel categories because a large number of mask annotations is required. We introduce ShapeMask, which learns the intermediate concept of object shape to address the problem of generalization in instance segmentation to novel categories. Shape-Mask starts with a bounding box detection and gradually refines it by first estimating the shape of the detected object through a collection of shape priors. Next, ShapeMask refines the coarse shape into an instance level mask by learning instance embeddings. The shape priors provide a strong cue for object-like prediction, and the instance embeddings model the instance specific appearance information. Shape-Mask significantly outperforms the state-of-the-art by 6.4 and 3.8 AP when learning across categories, and obtains competitive performance in the fully supervised setting. It is also robust to inaccurate detections, decreased model capacity, and small training data. Moreover, it runs efficiently with 150ms inference time and trains within 11 hours on TPUs. With a larger backbone model, ShapeMask increases the gap with state-of-the-art to 9.4 and 6.2 AP across categories. Code will be released.

show abstract

“…To cope with this, one popular approach is to introduce a multi-stage pipeline with object proposals [9,4,16,7]. Another approach is to train a recurrent network end-to-end with a custom loss function that outputs instances sequentially [14,18,17].…”

Section: Introductionmentioning

confidence: 99%

Semantic Instance Segmentation for Autonomous Driving

Brabandere

Neven

Gool

2017

2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

210

View full text Add to dashboard Cite

Semantic instance segmentation remains a challenge. We propose to tackle the problem with a discriminative loss function, operating at pixel level, that encourages a convolutional network to produce a representation of the image that can easily be clustered into instances with a simple post-processing step. Our approach of combining an offthe-shelf network with a principled loss function inspired by a metric learning objective is conceptually simple and distinct from recent efforts in instance segmentation and is well-suited for real-time applications. In contrast to previous works, our method does not rely on object proposals or recurrent mechanisms and is particularly well suited for tasks with complex occlusions. A key contribution of our work is to demonstrate that such a simple setup without bells and whistles is effective and can perform on-par with more complex methods. We achieve competitive performance on the Cityscapes segmentation benchmark.

show abstract

Multi-instance object segmentation with occlusion handling

Cited by 84 publications

References 37 publications

Learning Semantics-aware Distance Map with Semantics Layering Network for Amodal Instance Segmentation

Learning Semantics-aware Distance Map with Semantics Layering Network for Amodal Instance Segmentation

ShapeMask: Learning to Segment Novel Objects by Refining Shape Priors

Semantic Instance Segmentation for Autonomous Driving

Contact Info

Product

Resources

About