Saliency guided deep network for weakly-supervised image segmentation

Abstract

Weakly-supervised image segmentation is an important task in computer vision. A key problem is how to obtain high quality objects location from image-level category. Classification activation mapping is a common method which can be used to generate high-precise object location cues. However these location cues are generally very sparse and small such that they can not provide effective information for image segmentation. In this paper, we propose a saliency guided image segmentation network to resolve this problem. We employ a self-attention saliency method to generate subtle saliency maps, and render the location cues grow as seeds by seeded region growing method to expand pixel-level labels extent. In the process of seeds growing, we use the saliency values to weight the similarity between pixels to control the growing. Therefore saliency information could help generate discriminative object regions, and the effects of wrong salient pixels can be suppressed efficiently. Experimental results on a common segmentation dataset PASCAL VOC2012 demonstrate the effectiveness of our method.

Saliency Guided Self-Attention Network for Weakly and Semi-Supervised Semantic Segmentation

“…A group of approaches take the class activation maps (CAMs) [11] generated from classification networks as initial seeds. Since CAMs only focus on small discriminative regions which are too sparse to effectively supervise a segmentation model, various techniques such as adversarial erasing [12], [17], [21], [18] and region growing [13], [22] have been developed to expand sparse object seeds. Another research line introduces dilated convolutions of different rates [14], [16], [15], [23] to enlarge receptive fields in classification networks and aggregate multiple attention maps to achieve dense localization cues.…”

“…These methods significantly boost the segmentation performance, but all of them perform under full supervision. Although [22] utilized the self-attention scheme for WSSS, they only used this scheme to learn a saliency detector that is trained also in a fully-supervised manner. In this work, we apply the self-attention scheme to a weaklysupervised scenario which is more challenging.…”

Unsupervised Domain Adaptation for Semantic Segmentation of Urban Scenes

“…Semantic segmentation architectures are typically trained on huge datasets with pixelwise annotations (e.g., the Cityscapes [5] or CamVid [1] datasets), which are highly expensive, time-consuming and error-prone to generate. To overcome this issue, semisupervised methods are emerging, trying to exploit weakly annotated data (e.g., with only image labels or only bounding boxes) [25,31,37,39,13,6,14,32] or completely unlabeled [24,29,15,31,19] data after a first stage of supervised training. In particular the works of [22,31] have paved the way respectively to adversarial learning approaches for the semantic segmentation task and to their application to semi-supervised learning.…”

Adversarial Learning and Self-Teaching Techniques for Domain Adaptation in Semantic Segmentation

“…Region growing techniques have been recently applied to domain adaptation in semantic segmentation [26], [27]. In particular in [26] a semantic segmentation network is trained to segment the discriminative regions first and to progressively increase the pixel-level supervision by seeded region growing [55].…”

“…In particular in [26] a semantic segmentation network is trained to segment the discriminative regions first and to progressively increase the pixel-level supervision by seeded region growing [55]. In [27] the authors propose a saliency guided weaklysupervised segmentation network which utilizes salient information as guidance to help weakly segmentation through a seeded region growing procedure. In [56] the region growing problem is defined as a Markov Decision Process.…”