One-Shot Learning for Semantic Segmentation

Shaban, Amirreza; Bansal, Shray; Liu, Zhen; Essa, Irfan; Boots, Byron

doi:10.5244/c.31.167

Cited by 503 publications

(456 citation statements)

References 27 publications

Supporting

Mentioning

438

Contrasting

Unclassified

Order By: Relevance

“…For evaluation, we use two datasets: (a) PASCAL-5 i which combines images from the PASCAL VOC 2012 [7] and Extended SDS [11] datasets; and (b) COCO-20 i which is based on the MSCOCO dataset [16]. For PASCAL-5 i , we use the same 4-fold cross-validation setup as prior work [26,20,6]. Specifically, from the 20 object classes in PASCAL VOC 2012, for each fold i = 0, ..., 3, we sample five as test classes, and use the remaining 15 classes for training.…”

Section: Methodsmentioning

confidence: 99%

“…Metrics. As in [26,20,6], we use the mean intersectionover-union (mIoU) for quantitative evaluation. IoU of class l is defined as IoU l = T P l T P l +F P l +F N l , where T P, F P and F N are the number of pixels that are true positives, false positives and false negatives of the predicted segmentation masks, respectively.…”

Section: Methodsmentioning

confidence: 99%

“…In addition, we consider two alternative neural networks -VGG 16 and ResNet 101 -as the CNN for extracting image features. VGG 16 has also been used in prior work [26,20,6]. We also compare with an approach called Upper-bound that represents a variant of our full approach B+C1+C2 trained such that both training and testing datasets consist of the same classes.…”

Section: Methodsmentioning

confidence: 99%

“…Recently, prior work has addressed this problem by training an object segmenter on a large training set, under the few-shot constraint [26,6,20]. The training set is split into many small subsets.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Feature Weighting and Boosting for Few-Shot Segmentation

Nguyen

Todorovic

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

297

173

View full text Add to dashboard Cite

This paper is about few-shot segmentation of foreground objects in images. We train a CNN on small subsets of training images, each mimicking the few-shot setting. In each subset, one image serves as the query and the other(s) as support image(s) with ground-truth segmentation. The CNN first extracts feature maps from the query and support images. Then, a class feature vector is computed as an average of the support's feature maps over the known foreground. Finally, the target object is segmented in the query image by using a cosine similarity between the class feature vector and the query's feature map. We make two contributions by: (1) Improving discriminativeness of features so their activations are high on the foreground and low elsewhere; and (2) Boosting inference with an ensemble of experts guided with the gradient of loss incurred when segmenting the support images in testing. Our evaluations on the PASCAL-5 i and COCO-20 i datasets demonstrate that we significantly outperform existing approaches.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Feature Weighting and Boosting for Few-Shot Segmentation

Nguyen

Todorovic

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

297

173

View full text Add to dashboard Cite

show abstract

“…Segmentation masks with dashed border denote ground truth annotations. classification [25,23,24,18,6,20,12,14] and a few targeting at segmentation tasks [21,17,4,28,4,8].…”

Section: Introductionmentioning

confidence: 99%

PANet: Few-Shot Image Semantic Segmentation With Prototype Alignment

Wang

Liew²,

Zou

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

1,030

724

View full text Add to dashboard Cite

Despite the great progress made by deep CNNs in image semantic segmentation, they typically require a large number of densely-annotated images for training and are difficult to generalize to unseen object categories. Few-shot segmentation has thus been developed to learn to perform segmentation from only a few annotated examples. In this paper, we tackle the challenging few-shot segmentation problem from a metric learning perspective and present PANet, a novel prototype alignment network to better utilize the information of the support set. Our PANet learns classspecific prototype representations from a few support images within an embedding space and then performs segmentation over the query images through matching each pixel to the learned prototypes. With non-parametric metric learning, PANet offers high-quality prototypes that are representative for each semantic class and meanwhile discriminative for different classes. Moreover, PANet introduces a prototype alignment regularization between support and query. With this, PANet fully exploits knowledge from the support and provides better generalization on few-shot segmentation. Significantly, our model achieves the mIoU score of 48.1% and 55.7% on PASCAL-5 i for 1-shot and 5-shot settings respectively, surpassing the state-of-the-art method by 1.8% and 8.6%.

show abstract