Weakly-supervised Instance Segmentation via Class-agnostic Learning with Salient Images

Wang, Xinggang; Feng, Jiapei; Hu, Bin; Ding, Qi; Ran, Longjin; Chen, Xiaoxin; Liu, Wenyu

doi:10.1109/cvpr46437.2021.01009

Cited by 32 publications

(25 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our method outperforms BBTP [16] by 12.3% AP with the same backbone. In contrast to the recent box-supervised methods, our method outperforms BBAM [26] and BoxCaseg [47] by 7.7% AP and 2.5% AP using ResNet-101. It achieves 33.4% AP and 35.4% AP, which is higher than BoxInst [44] by 0.2% and 0.4% with ResNet-101 and ResNet-101-DCN backbones, respectively.…”

Section: Resultsmentioning

confidence: 66%

“…To deal with this problem, box-supervised instance segmentation takes advantage of the simple box annotation rather than the pixel-wise mask labels, which has recently attracted a lot of research attentions [16, 24-26, 44, 47]. To enable pixel-wise supervision with box annotation, some methods [26,47] focus on generating the pseudo mask labels by an independent network, which needs to employ extra auxiliary salient data [47] or post-processing methods like MCG [39] and CRF [23] to obtain precise pseudo labels. Due to the involved multiple separate steps, the training pipeline becomes complicated with many hyper-parameters.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Box-supervised Instance Segmentation with Level Set Evolution

Li¹,

Liu²,

Zhu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

In contrast to the fully supervised methods using pixel-wise mask labels, box-supervised instance segmentation takes advantage of the simple box annotations, which has recently attracted a lot of research attentions. In this paper, we propose a novel single-shot box-supervised instance segmentation approach, which integrates the classical level set model with deep neural network delicately. Specifically, our proposed method iteratively learns a series of level sets through a continuous Chan-Vese energy-based function in an end-to-end fashion. A simple mask supervised SOLOv2 model is adapted to predict the instance-aware mask map as the level set for each instance. Both the input image and its deep features are employed as the input data to evolve the level set curves, where a box projection function is employed to obtain the initial boundary. By minimizing the fully differentiable energy function, the level set for each instance is iteratively optimized within its corresponding bounding box annotation. The experimental results on four challenging benchmarks demonstrate the leading performance of our proposed approach to robust instance segmentation in various scenarios. The code is available at: https://github.com/LiWentomng/boxlevelset.

show abstract

Section: Resultsmentioning

confidence: 66%

Section: Introductionmentioning

confidence: 99%

Box-supervised Instance Segmentation with Level Set Evolution

Li¹,

Liu²,

Zhu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…In this case, the position and the number of relevant objects can be estimated, which further contributes to proper characterization of the site from remote. The present version of AerialWaste can support the training and testing of weakly supervised localization (WSL) architectures 33,34 by providing segmentation masks for the images in the test set. Given the high cost of creating object-level masks 34 we have added to the dataset a number of segmentation masks sufficient to evaluate a WSL model trained with the provided whole-image labels of the object types.…”

Section: Supported Use Cases and Extensionmentioning

confidence: 99%

AerialWaste dataset for landfill discovery in aerial and satellite images

Torres

Fraternali

2023

Sci Data

View full text Add to dashboard Cite

Illegal landfills are sites where garbage is dumped violating waste management laws. Aerial images enable the use of photo interpretation for territory scanning and landfill detection but this practice is hindered by the manual nature of this task which also requires expert knowledge. Deep Learning methods can help capture the analysts’ expertise and build automated landfill discovery tools. However, this goal requires public high-quality datasets for model training and testing. At present no such datasets exist and this gap penalizes the research toward scalable and accurate landfill discovery methods. We present a dataset for landfill detection featuring airborne, WorldView-3, and GoogleEarth images annotated by professional photo interpreters. It comprises 3,478 positive and 6,956 negative examples. Most positive instances are characterized by metadata: the type of waste, its storage mode, the type of the site, and the evidence and severity of the illicit. The dataset has been technically validated by building an accurate landfill detector and is accompanied by a visualization and annotation tool.

show abstract

“…One of the most popular ways to increase the number of segmentation classes is partially-supervised learning. It utilizes weak image-level [15][16][17] or box-level [18][19][20][21][22][23] supervision to segment objects that have no mask annotations, thus lowering the annotation costs. Despite the successes of partially-supervised methods, they can only segment the classes covered by the image/box-level annotation and not a wide general range of novel classes.…”

Section: Introductionmentioning

confidence: 99%

Open-Vocabulary Instance Segmentation via Robust Cross-Modal Pseudo-Labeling

Huynh¹,

Kuen²,

Lin³

et al. 2021

Preprint

View full text Add to dashboard Cite

Open-vocabulary instance segmentation aims at segmenting novel classes without mask annotations. It is an important step toward reducing laborious human supervision. Most existing works first pretrain a model on captioned images covering many novel classes and then finetune it on limited base classes with mask annotations. However, the high-level textual information learned from caption pretraining alone cannot effectively encode the details required for pixel-wise segmentation. To address this, we propose a cross-modal pseudo-labeling framework, which generates training pseudo masks by aligning word semantics in captions with visual features of object masks in images. Thus, our framework is capable of labeling novel classes in captions via their word semantics to self-train a student model. To account for noises in pseudo masks, we design a robust student model that selectively distills mask knowledge by estimating the mask noise levels, hence mitigating the adverse impact of noisy pseudo masks. By extensive experiments, we show the effectiveness of our framework, where we significantly improve mAP score by 4.5% on MS-COCO and 5.1% on the large-scale Open Images & Conceptual Captions datasets compared to the state-of-the-art.

show abstract

Weakly-supervised Instance Segmentation via Class-agnostic Learning with Salient Images

Cited by 32 publications

References 43 publications

Box-supervised Instance Segmentation with Level Set Evolution

Box-supervised Instance Segmentation with Level Set Evolution

AerialWaste dataset for landfill discovery in aerial and satellite images

Open-Vocabulary Instance Segmentation via Robust Cross-Modal Pseudo-Labeling

Contact Info

Product

Resources

About