Learning to Segment via Cut-and-Paste

Remez, Tal; Huang, Jonathan; Brown, Matthew A.

doi:10.1007/978-3-030-01234-2_3

Cited by 88 publications

(64 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thanks to low annotation costs of weak labels, approaches in this category can utilize more training images of diverse objects, although they have to compensate for missing information in weak labels. For instance segmentation, bounding boxes have been widely used as weak labels since they provide every property of objects except shape [24,44]. However, it is still costly to obtain box labels for a variety of classes in a large number of images as they are manually annotated.…”

Section: Introductionmentioning

confidence: 99%

Weakly Supervised Learning of Instance Segmentation With Inter-Pixel Relations

Ahn

Cho

Kwak

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

525

585

View full text Add to dashboard Cite

This paper presents a novel approach for learning instance segmentation with image-level class labels as supervision. Our approach generates pseudo instance segmentation labels of training images, which are used to train a fully supervised model. For generating the pseudo labels, we first identify confident seed areas of object classes from attention maps of an image classification model, and propagate them to discover the entire instance areas with accurate boundaries. To this end, we propose IRNet, which estimates rough areas of individual instances and detects boundaries between different object classes. It thus enables to assign instance labels to the seeds and to propagate them within the boundaries so that the entire areas of instances can be estimated accurately. Furthermore, IRNet is trained with interpixel relations on the attention maps, thus no extra supervision is required. Our method with IRNet achieves an outstanding performance on the PASCAL VOC 2012 dataset, surpassing not only previous state-of-the-art trained with the same level of supervision, but also some of previous models relying on stronger supervision.

show abstract

Section: Introductionmentioning

confidence: 99%

Weakly Supervised Learning of Instance Segmentation With Inter-Pixel Relations

Ahn

Cho

Kwak

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

525

585

View full text Add to dashboard Cite

show abstract

“…Combining different sources of synthetic data with real data, and then generating a realistic composition of the both, has been successfully applied to various tasks, such as semisupervised foreground-background segmentation (Remez, Huang, and Brown 2018;Alhaija et al 2018;Dwibedi, Misra, and Hebert 2017), object detection (Dvornik, Mairal, and Schmid 2018;Dwibedi, Misra, and Hebert 2017) or 3d object pose estimation (Alhaija et al 2018). The two cut-and-paste methods (Dwibedi, Misra, and Hebert 2017;Remez, Huang, and Brown 2018) use simple blending techniques, and only for the foreground object, while we propose to learn a blending for both the background and foreground, accordingly. Note that while (Alhaija et al 2018) and (Dwibedi, Misra, and Hebert 2017) take the 3d geometry of the scene into account, they only consider 3d rigid objects.…”

Section: General Data Augmentation Through Image Synthesismentioning

confidence: 99%

Human Synthesis and Scene Compositing

Zanfir

Oneata

Popa

et al. 2020

AAAI

View full text Add to dashboard Cite

Generating good quality and geometrically plausible synthetic images of humans with the ability to control appearance, pose and shape parameters, has become increasingly important for a variety of tasks ranging from photo editing, fashion virtual try-on, to special effects and image compression. In this paper, we propose a HUSC (HUman Synthesis and Scene Compositing) framework for the realistic synthesis of humans with different appearance, in novel poses and scenes. Central to our formulation is 3d reasoning for both people and scenes, in order to produce realistic collages, by correctly modeling perspective effects and occlusion, by taking into account scene semantics and by adequately handling relative scales. Conceptually our framework consists of three components: (1) a human image synthesis model with controllable pose and appearance, based on a parametric representation, (2) a person insertion procedure that leverages the geometry and semantics of the 3d scene, and (3) an appearance compositing process to create a seamless blending between the colors of the scene and the generated human image, and avoid visual artifacts. The performance of our framework is supported by both qualitative and quantitative results, in particular state-of-the art synthesis scores for the DeepFashion dataset.

show abstract

“…These methods typically rely on bottom-up segment proposals [6] [7]. In contrast with this approach, [8] proposes an adversarial scheme that learns to segment without using any object proposal technique. Although these works tackle weakly-supervised instance segmentation, their weak supervision consists in using bounding boxes, thus their main challenge resides in how to separate the foreground from the background within a bounding box.…”

Section: Related Workmentioning

confidence: 99%

Mask-guided sample selection for semi-supervised instance segmentation

Bellver

Salvador

Torres

et al. 2020

Multimed Tools Appl

View full text Add to dashboard Cite

Image segmentation methods are usually trained with pixel-level annotations, which require significant human effort to collect. The most common solution to address this constraint is to implement weakly-supervised pipelines trained with lower forms of supervision, such as bounding boxes or scribbles. Another option are semi-supervised methods, which leverage a large amount of unlabeled data and a limited number of strongly-labeled samples. In this second setup, samples to be strongly-annotated can be selected randomly or with an active learning mechanism that chooses the ones that will maximize the model performance. In this work, we propose a sample selection approach to decide which samples to annotate for semi-supervised instance segmentation. Our method consists in first predicting pseudo-masks for the unlabeled pool of samples, together with a score predicting the quality of the mask. This score is an estimate of the Intersection Over Union (IoU) of the segment with the ground truth mask. We study which samples are better to annotate given the quality score, and show how our approach outperforms a random selection, leading to improved performance for semi-supervised instance segmentation with low annotation budgets.

show abstract

Learning to Segment via Cut-and-Paste

Cited by 88 publications

References 24 publications

Weakly Supervised Learning of Instance Segmentation With Inter-Pixel Relations

Weakly Supervised Learning of Instance Segmentation With Inter-Pixel Relations

Human Synthesis and Scene Compositing

Mask-guided sample selection for semi-supervised instance segmentation

Contact Info

Product

Resources

About