ECS-Net: Improving Weakly Supervised Semantic Segmentation by Using Connections Between Class Activation Maps

Sun, Kunyang; Shi, Haoqing; Zhang, Zhengming; Huang, Yongming

doi:10.1109/iccv48922.2021.00719

Cited by 92 publications

(19 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…4 we compare contour F-score with our baseline SEAM [4] and the best region similarity method PMM [38], where the contour quality of our method is superior. Furthermore, if we do as in the well-established DAVIS benchmark for video object segmentation [33] and consider the the two metrics equally, we outright outperform both meth- [42] 64.9 65.5 --SEAM [4] 64.5 65.7 --CONTA [43] 66.1 66.7 33.4 -CDA [44] 66.1 66.8 33.7 -MCIS [45] 66.2 66.9 --ECS-Net [46] 66.6 67.6 --CGNet [47] 68.4 68.2 36.4 -CPN [48] 67.8 68.5 --PMM [38] 68. ods. Thus, in applications where contour quality bears even a small importance, our method is preferable.…”

Section: State-of-the-art Comparisonmentioning

confidence: 99%

Activation-Based Sampling for Pixel- to Image-Level Aggregation in Weakly-Supervised Segmentation

Arvi¹,

Felsberg²,

Zhang³

2022

Preprint

View full text Add to dashboard Cite

Classification networks can be used to localize and segment objects in images by means of class activation maps (CAMs). However, without pixel-level annotations, they are known to (1) mainly focus on discriminative regions, and (2) to produce diffuse CAMs without well-defined prediction contours. In this work, we approach both problems with two contributions for improving CAM learning. First, we incorporate importance sampling based on the classwise probability mass function induced by the CAMs to produce stochastic image-level class predictions. This results in CAMs which activate over a larger extent of the objects. Second, we formulate a feature similarity loss term which aims to match the prediction contours with edges in the image. As a third contribution, we conduct experiments on the PASCAL VOC and MS-COCO benchmark datasets to demonstrate that these modifications significantly increase the performance in terms of contour accuracy, while being comparable to current state-of-the-art methods in terms of region similarity.

show abstract

Section: State-of-the-art Comparisonmentioning

confidence: 99%

Activation-Based Sampling for Pixel- to Image-Level Aggregation in Weakly-Supervised Segmentation

Arvi¹,

Felsberg²,

Zhang³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Erasure and accumulation. Erasure methods explore more object regions by intentionally removing the discriminative regions from the images [23,37,43] or feature maps [8,18]. However, erasing most of the discriminative regions may confuse the classifier and result in false positives.…”

Section: Related Workmentioning

confidence: 99%

“…1 presents the comparison with other advanced methods on PASCAL VOC 2012 train set. Among these compared methods, ECS [37] provides the best results with a mIoU of 56.6%. Our proposed SIPE achieves the state-of-the-art performance of 58.6%.…”

Section: Comparison With State-of-the-artsmentioning

confidence: 99%

Self-supervised Image-specific Prototype Exploration for Weakly Supervised Semantic Segmentation

Chen¹,

Yang²,

Lai³

et al. 2022

Preprint

View full text Add to dashboard Cite

Weakly Supervised Semantic Segmentation (WSSS) based on image-level labels has attracted much attention due to low annotation costs. Existing methods often rely on Class Activation Mapping (CAM) that measures the correlation between image pixels and classifier weight. However, the classifier focuses only on the discriminative regions while ignoring other useful information in each image, resulting in incomplete localization maps. To address this issue, we propose a Self-supervised Image-specific Prototype Exploration (SIPE) that consists of an Image-specific Prototype Exploration (IPE) and a General-Specific Consistency (GSC) loss. Specifically, IPE tailors prototypes for every image to capture complete regions, formed our Image-Specific CAM (IS-CAM), which is realized by two sequential steps. In addition, GSC is proposed to construct the consistency of general CAM and our specific IS-CAM, which further optimizes the feature representation and empowers a self-correction ability of prototype exploration. Extensive experiments are conducted on PASCAL VOC 2012 and MS COCO 2014 segmentation benchmark and results show our SIPE achieves new state-of-the-art performance using only image-level labels. The code is available at https://github.com/chenqi1126/SIPE.

show abstract

“…To address this issue, we use the latest non-empty prototype for each class. For the segmentation network, we use Deeplab [4] with ResNet38 backbone as in [2,25,34,43,60].…”

Section: Implementation Detailsmentioning

confidence: 99%

“…To further improve the quality of pseudo labels, we also apply a commonly used Random Walk (RW) approach [2] as in [3,28,43,49,60]. After applying the RW to our CAMs, the performance of pseudo labels achieve 71.0% mIoU on PASCAL VOC 2012 train set.…”

Section: Comparison With State-of-the-artsmentioning

confidence: 99%

Exploring Pixel-level Self-supervision for Weakly Supervised Semantic Segmentation

Yoon¹,

Kweon²,

Jeong³

et al. 2021

Preprint

View full text Add to dashboard Cite

Existing studies in weakly supervised semantic segmentation (WSSS) have utilized class activation maps (CAMs) to localize the class objects. However, since a classification loss is insufficient for providing precise object regions, CAMs tend to be biased towards discriminative patterns (i.e., sparseness) and do not provide precise object boundary information (i.e., impreciseness). To resolve these limitations, we propose a novel framework (composed of MainNet and SupportNet.) that derives pixel-level selfsupervision from given image-level supervision. In our framework, with the help of the proposed Regional Contrastive Module (RCM) and Multi-scale Attentive Module (MAM), MainNet is trained by self-supervision from the SupportNet. The RCM extracts two forms of selfsupervision from SupportNet: (1) class region masks generated from the CAMs and (2) class-wise prototypes obtained from the features according to the class region masks. Then, every pixel-wise feature of the MainNet is trained by the prototype in a contrastive manner, sharpening the resulting CAMs. The MAM utilizes CAMs inferred at multiple scales from the SupportNet as self-supervision to guide the MainNet. Based on the dissimilarity between the multiscale CAMs from MainNet and SupportNet, CAMs from the MainNet are trained to expand to the less-discriminative regions. The proposed method shows state-of-the-art WSSS performance both on the train and validation sets on the PASCAL VOC 2012 dataset. For reproducibility, code will be available publicly soon.

show abstract

ECS-Net: Improving Weakly Supervised Semantic Segmentation by Using Connections Between Class Activation Maps

Cited by 92 publications

References 33 publications

Activation-Based Sampling for Pixel- to Image-Level Aggregation in Weakly-Supervised Segmentation

Activation-Based Sampling for Pixel- to Image-Level Aggregation in Weakly-Supervised Segmentation

Self-supervised Image-specific Prototype Exploration for Weakly Supervised Semantic Segmentation

Exploring Pixel-level Self-supervision for Weakly Supervised Semantic Segmentation

Contact Info

Product

Resources

About