Non-Salient Region Object Mining for Weakly Supervised Semantic Segmentation

Yao, Yazhou; Chen, Tao; Xie, Guo-Sen; Zhang, Chuanyi; Shen, Fumin; Wu, Qi; Tang, Zhenmin; Zhang, Jian

doi:10.1109/cvpr46437.2021.00265

Cited by 170 publications

(72 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Specifically, Ours-L achieves 69.2% and 70.6% mIoU on the PASCAL VOC val set with DeepLabV2 initialized with ImageNet and MS COCO pre-trained weights, respectively, which recover 90.7% and 91.0% of the upper bound of their fully-supervised counterparts. Our methods also achieve comparable performance with recent state-of-the-art WSSS methods us-ing extra saliency maps, such as NSROM (Yao et al, 2021), DRS (Kim et al, 2021), EPS (Lee et al, 2021c), AuxSegNet (Xu et al, 2021), and EDAM (Wu et al, 2021). Our method also outperforms recent methods with superior backbone networks, such as PMM (Li et al, 2021b), which uses Res2Net101 (Gao et al, 2021) as the backbone for semantic segmentation.…”

Section: Imagementioning

confidence: 57%

“…proposed OAA, which accumulated the activated regions during the different training stage. In (Yao et al, 2021), Yao et al . proposed a graph reasoning and non-salient region mining module to capture more object extents from non-salient regions, since the saliency prior used in OAA did not always correspond to the foreground objects.…”

Section: Wsss With Image-level Labelsmentioning

confidence: 99%

See 1 more Smart Citation

Weakly-Supervised Semantic Segmentation with Visual Words Learning and Hybrid Pooling

Ru¹,

Du²,

Zhan³

et al. 2022

Preprint

View full text Add to dashboard Cite

Weakly-Supervised Semantic Segmentation (WSSS) methods with image-level labels generally train a classification network to generate the Class Activation Maps (CAMs) as the initial coarse segmentation labels. However, current WSSS methods still perform far from satisfactorily because their adopted CAMs 1) typically focus on partial discriminative object regions and 2) usually contain useless background regions. These two problems are attributed to the sole image-level supervision and aggregation of global information when training the classification networks. In this work, we propose the visual words learning module and hybrid pooling approach, and incorporate them in the classification network to mitigate the above problems. In the visual words learning module, we counter the first problem by enforcing the classification network to learn fine-grained visual word labels so that more object extents could be discovered. Specifically, the visual words are learned with a codebook, which could be updated via two proposed strategies, i.e. learning-based strategy and memory-bank strategy. The second drawback of CAMs is alleviated with the proposed hybrid pooling, which incorporates the global average and local discriminative information to simultaneously ensure object completeness and reduce background regions. We

show abstract

Section: Imagementioning

confidence: 57%

Section: Wsss With Image-level Labelsmentioning

confidence: 99%

Weakly-Supervised Semantic Segmentation with Visual Words Learning and Hybrid Pooling

Ru¹,

Du²,

Zhan³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…As the basic task of scene understanding, semantic segmentation technology based on pixel-by-pixel classification has been widely studied [22][23][24]. Many semantic segmentation methods based on deep learning have been proposed [25][26][27][28]. Currently, there are four main types of networks, namely, the fully convolutional network (FCN) [29], the convolutional neural network (CNN) [30], the recurrent neural network (RNN) [31], and the generative adversarial network (GAN) [32].…”

Section: Semantic Segmentation Based On Deep Learningmentioning

confidence: 99%

MAGNet: A camouflaged object detection network simulating the observation effect of a magnifier

Jiang

Cai

Jiang

et al. 2022

Preprint

View full text Add to dashboard Cite

In recent years, protecting important objects by simulating animal camouflage has been widely used in many fields. Therefore, camouflaged object detection (COD) technology has emerged. COD is more difficult than traditional object detection techniques because of the high degree of fusion of camouflaged objects with the background. In this paper, we strive to more accurately identify camouflaged objects. Inspired by the use of magnifiers to search for hidden objects in pictures, we propose a COD network that simulates the observation effect of a magnifier, called the MAGnifier Network (MAGNet). Specifically, our MAGNet contains two parallel modules, i.e., the ergodic magnification module (EMM) and the attention focus module (AFM). The EMM is designed to mimic the process of a magnifier enlarging an image, and AFM is used to simulate the observation process in which human attention is highly focused on a region. The two sets of output camouflaged object maps are merged to simulate the observation of an object by a magnifier. In addition, a weighted key point area perception loss function, which is more applicable to COD, is designed based on two modules to give higher attention to the camouflaged object. Extensive experiments demonstrate that compared with 14 cutting-edge detection models, MAGNet can achieve the best comprehensive effect on eight evaluation metrics on the public COD dataset, and the segmentation accuracy is significantly improved. We also validate the models' generalization ability on a military camouflaged object dataset constructed in-house. Finally, we experimentally explore some extended applications of COD.

show abstract

“…As an additional guidance for network to pay attention to the entire region of objects, some existing works attempt to devise auxiliary tasks such as sub-category classification [3], self-equivariant regularization with scale variance minimization [49], class-wise co-attention extraction [33,42], anti-adversarial attack [28], and complementary patch loss [60]. Many WSSS methods [15,16,21,30,33,42,55,56] have been proposed to employ the pre-trained saliency detection module, which distinguishes dominant foreground object from its background, as a complementary source of information for enhancing CAMs and generating precise pseudo-pixel labels.…”

Section: Related Workmentioning

confidence: 99%

Exploring Pixel-level Self-supervision for Weakly Supervised Semantic Segmentation

Yoon¹,

Kweon²,

Jeong³

et al. 2021

Preprint

View full text Add to dashboard Cite

Existing studies in weakly supervised semantic segmentation (WSSS) have utilized class activation maps (CAMs) to localize the class objects. However, since a classification loss is insufficient for providing precise object regions, CAMs tend to be biased towards discriminative patterns (i.e., sparseness) and do not provide precise object boundary information (i.e., impreciseness). To resolve these limitations, we propose a novel framework (composed of MainNet and SupportNet.) that derives pixel-level selfsupervision from given image-level supervision. In our framework, with the help of the proposed Regional Contrastive Module (RCM) and Multi-scale Attentive Module (MAM), MainNet is trained by self-supervision from the SupportNet. The RCM extracts two forms of selfsupervision from SupportNet: (1) class region masks generated from the CAMs and (2) class-wise prototypes obtained from the features according to the class region masks. Then, every pixel-wise feature of the MainNet is trained by the prototype in a contrastive manner, sharpening the resulting CAMs. The MAM utilizes CAMs inferred at multiple scales from the SupportNet as self-supervision to guide the MainNet. Based on the dissimilarity between the multiscale CAMs from MainNet and SupportNet, CAMs from the MainNet are trained to expand to the less-discriminative regions. The proposed method shows state-of-the-art WSSS performance both on the train and validation sets on the PASCAL VOC 2012 dataset. For reproducibility, code will be available publicly soon.

show abstract

Non-Salient Region Object Mining for Weakly Supervised Semantic Segmentation

Cited by 170 publications

References 47 publications

Weakly-Supervised Semantic Segmentation with Visual Words Learning and Hybrid Pooling

Weakly-Supervised Semantic Segmentation with Visual Words Learning and Hybrid Pooling

MAGNet: A camouflaged object detection network simulating the observation effect of a magnifier

Exploring Pixel-level Self-supervision for Weakly Supervised Semantic Segmentation

Contact Info

Product

Resources

About