Reliability Does Matter: An End-to-End Weakly Supervised Semantic Segmentation Approach

Zhang, Bingfeng; Xiao, Jimin; Wei, Yunchao; Sun, Mingjie; Huang, Kaizhu

doi:10.1609/aaai.v34i07.6971

Cited by 181 publications

(151 citation statements)

References 31 publications

Supporting

Mentioning

150

Contrasting

Order By: Relevance

“…To alleviate the problem caused by various scales of buildings in HR images, the multi-scale aggregation strategy [50] is used to generate CAM with higher generality. Specifically, given an image and scale ratio, multi-scale CAM is generated by taking average of CAMs from different scaled images as shown below:…”

Section: ) Generating Pseudo-masksmentioning

confidence: 99%

On the Effectiveness of Weakly Supervised Semantic Segmentation for Building Extraction From High-Resolution Remote Sensing Imagery

Zhang

Xiao

et al. 2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

A critical obstacle to achieve semantic segmentation of remote sensing images by the deep convolutional neural network is the requirement of huge pixel-level labels. Taking building extraction as an example, this study focuses on how to effectively apply weakly supervised semantic segmentation (WSSS) to highresolution remote sensing (HR) images with image-level labels, which is a prominent solution for the huge labeling challenge. The widely-used two-step WSSS framework is adopted, in which the pseudo-masks are first produced from image-level labels and followed by a segmentation network trained by the pseudo-masks. In addition, the fully connected conditional random field (CRF) is utilized to explore spatial context in both training and prediction stages. Detailed analyses are implemented on applying WSSS on HR images in terms of producing pseudo-masks, training segmentation network, and optimizing predictions. We show that the trade-off between precision and recall of pseudo-masks, as well as the boundary accuracy and the background, needs to be carefully considered. The benefits of the segmentation network in the two-step framework are demonstrated in comparison to using classification network only for WSSS, and the effects of CRF-loss are identified to be powerful for improving the segmentation network while not appropriate for dense buildings. An overlapping strategy and CRF post-processing are further demonstrated to be effective for optimizing the segmentation results during inferencing. Through deliberate settings, we can generate results comparable to fully supervised on the ISPRS Potsdam and Vaihingen dataset, which is meaningful for promoting WSSS applications for extracting geographic information from HR images.

show abstract

Section: ) Generating Pseudo-masksmentioning

confidence: 99%

On the Effectiveness of Weakly Supervised Semantic Segmentation for Building Extraction From High-Resolution Remote Sensing Imagery

Zhang

Xiao

et al. 2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

show abstract

“…There are also other approaches [14] focusing on exploring the relationship between pixels. RRM [25] proposes a one-step solution to handle the task and also put forward the two-step version which gets a new state-of-the-art performance. The method also considers using multi-scale original images to generate the better seed.…”

Section: Pixel-level Semantic Segmentation With Image-level Annotationsmentioning

confidence: 99%

“…In Table 2 [2,4,5,[11][12][13][14]17,24,25,[33][34][35][36][37][38], we compared our approach with other recently introduced weaklysupervised semantic segmentation methods on the PASCAL VOC validation and test sets. Besides, we give the results of two early fully-supervised methods.…”

Section: Comparisons To the State-of-the-artsmentioning

confidence: 99%

See 1 more Smart Citation

Deep graph cut network for weakly-supervised semantic segmentation

Feng

Wang

Liu

2021

Sci. China Inf. Sci.

View full text Add to dashboard Cite

The scarcity of fully-annotated data becomes the biggest obstacle that prevents many deep learning approaches from widely applied. Weakly-supervised visual learning which can utilize inexact annotations is developed rapidly to remedy such a situation. In this paper, we study the weakly-supervised task achieving pixel-level semantic segmentation only with image-level labels as supervision. Different from other methods, our approach tries to transform the weakly-supervised visual learning problem into a semi-supervised visual learning problem and then utilizes semi-supervised learning methods to solve it. Utilizing this transformation, we can adopt effective semi-supervised methods to perform transductive learning with context information. In the semi-supervised learning module, we propose to use the graph cut algorithm to label more supervision from the activation seeds generated from a classification network. The generated labels can provide the segmentation model with effective supervision information; moreover, the graph cut module can benefit from features extracted by the segmentation model. Then, each of them updates and optimizes the other iteratively until convergence. Experiment results on PASCAL VOC and COCO benchmarks demonstrate the effectiveness of the proposed deep graph cut algorithm for weakly-supervised semantic segmentation.

show abstract

“…Weakly supervised learning has made remarkable progress recently. Existing methods have been successfully applied on object localization [2,9,17,20,28,31,[33][34][35][36][37], object detection [3,10,23,30] and segmentation [1,6,8,13,[24][25][26]32]. Weakly Supervised Object Localization (WSOL) refers to mining information of object locations by only using image-level annotations.…”

Section: Introductionmentioning

confidence: 99%

Dual-Gradients Localization Framework for Weakly Supervised Object Localization

Tan

Ruan

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Weakly Supervised Object Localization (WSOL) aims to learn object locations in a given image while only using image-level annotations. For highlighting the whole object regions instead of the discriminative parts, previous works often attempt to train classification model for both classification and localization tasks. However, it is hard to achieve a good tradeoff between the two tasks, if only classification labels are employed for training on a single classification model. In addition, all of recent works just perform localization based on the last convolutional layer of classification model, ignoring the localization ability of other layers. In this work, we propose an offline framework to achieve precise localization on any convolutional layer of a classification model by exploiting two kinds of gradients, called Dual-Gradients Localization (DGL) framework. DGL framework is developed based on two branches: 1) Pixel-level Class Selection, leveraging gradients of the target class to identify the correlation ratio of pixels to the target class within any convolutional feature maps, and 2) Class-aware Enhanced Maps, utilizing gradients of classification loss function to mine entire target object regions, which would not damage classification performance. Extensive experiments on public ILSVRC and CUB-200-2011 datasets show the effectiveness of the proposed DGL framework. Especially, our DGL obtains a new state-of-the-art Top-1 localization error of 43.55% on the ILSVRC benchmark.

show abstract

Reliability Does Matter: An End-to-End Weakly Supervised Semantic Segmentation Approach

Cited by 181 publications

References 31 publications

On the Effectiveness of Weakly Supervised Semantic Segmentation for Building Extraction From High-Resolution Remote Sensing Imagery

On the Effectiveness of Weakly Supervised Semantic Segmentation for Building Extraction From High-Resolution Remote Sensing Imagery

Deep graph cut network for weakly-supervised semantic segmentation

Dual-Gradients Localization Framework for Weakly Supervised Object Localization

Contact Info

Product

Resources

About