Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast

Abstract

Recent attention has been devoted to the pursuit of learning semantic segmentation models exclusively from image tags, a paradigm known as image-level Weakly Supervised Semantic Segmentation (WSSS). Existing attempts adopt the Class Activation Maps (CAMs) as priors to mine object regions yet observe the imbalanced activation issue, where only the most discriminative object parts are located. In this paper, we argue that the distribution discrepancy between the discriminative and the non-discriminative parts of objects prevents the model from producing complete and precise pseudo masks as ground truths. For this purpose, we propose a Pixel-Level Domain Adaptation (PLDA) method to encourage the model in learning pixel-wise domain-invariant features. Specifically, a multi-head domain classifier trained adversarially with the feature extraction is introduced to promote the emergence of pixel features that are invariant with respect to the shift between the source (i.e., the discriminative object parts) and the target (i.e., the nondiscriminative object parts) domains. In addition, we come up with a Confident Pseudo-Supervision strategy to guarantee the discriminative ability of each pixel for the segmentation task, which serves as a complement to the intra-image domain adversarial training. Our method is conceptually simple, intuitive and can be easily integrated into existing WSSS methods. Taking several strong baseline models as instances, we experimentally demonstrate the effectiveness of our approach under a wide range of settings.

A Weakly Supervised Semantic Segmentation Method based on Local Superpixel Transformation

“…In Figure 6, we employ multiscale inference by integrating prediction results from images of varying complexity, a common practice in [3,52]. We evaluate the accuracy of our SPL and compare it to pseudo masks generated by other state-of-the-art weakly supervised semantic segmentation methods, including EPS [33], L2G [25], PPC (EPS) [14], and ReCAM [12]. Our method outperforms…”

“…71.9 recam(irn) [12] I. 70.9 PPC(EPS) [14] I.+S. the others in terms of accurately correcting semantic seeds at the pixel level during training, which leads to a more comprehensive understanding of the target object.…”

“…In Table 2, we compare the accuracy of pseudo masks generated by various methods on the PASCAL VOC dataset, with significance plots included for all methods except ReCAM [12]. Our method achieves a significant improvement over the second most effective method, PPC (EPS) [14], with a pseudo mask accuracy of 76.1%. Table 4 shows the segmentation scores of our method on the PASCAL VOC dataset when the backbone is VGG16 and the comparison with other methods.…”

“…This paper's method can be used in most of the current CAM-based pseudo mask generation processes. We conduct extensive experiments on current sota, including PPC(EPS) [14], PPC(SEAM) [14], L2G [25], ReCAM [12] (ResNet50), ReCAM(IRNet) [12], EPS [33] and compared them with the method in this paper, resulting in a comparison plot in Fig. 5.…”

Developing a Traffic Model to Estimate Vehicle Emissions: An Application in Seoul, Korea

“…SANCE [36] and ADELE [42] propose advanced pipelines to only remove minor noise in pseudo labels. In addition, some studies [35,25,13] employ saliency supervision to remove FP in pseudo labels. However, saliency supervision requires class-agnostic pixel-wise annotations and ignores small and low-prominent objects.…”