Joint Learning of Saliency Detection and Weakly Supervised Semantic Segmentation

Zeng, Yu; Zhuge, Yunzhi; Lu, Huchuan; Zhang, Lihe

doi:10.1109/iccv.2019.00732

Cited by 68 publications

(7 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several studies have utilized region erasing, 18 , 23 and region growing 19 , 24 to force the network to pay more attention to non-discriminatory regions. In addition, additional supervision has been introduced to close the gap, such as adding saliency maps, 2 , 25 – 28 CLIP supervision, 29 – 31 out-of-distribution data, 32 and contrastive learning 4 , 21 , 33 , 34 . References 21 and 35 narrow the gap by comparing pixel and prototype representations to provide pixel-level supervised signals.…”

Section: Related Workmentioning

confidence: 99%

TSD-CAM: transformer-based self distillation with CAM similarity for weakly supervised semantic segmentation

Yan,

Chen,

Tang

2024

J. Electron. Imag.

View full text Add to dashboard Cite

Weakly supervised semantic segmentation (WSSS) using only image-level labels is a challenging task. Most existing methods utilize class activation map (CAM) to generate pixel-level pseudo labels for supervised training. However, the gap between classification and segmentation hinders the network from obtaining more comprehensive semantic information and generating more accurate pseudo masks for segmentation. To address this issue, we propose TSD-CAM, a transformer-based self distillation (SD) method that utilizes CAM similarity. TSD-CAM uses the similarity between CAMs generated from different views as a distillation target, providing additional supervision for the network and narrowing the gap between classification and segmentation. SD supervision allows the network to acquire more semantic information and refine CAMs to generate higher precision pseudo-labels. In addition, we propose the adaptive pixel refinement module, which adaptively refines and adjusts images based on pixel variations, further improving the precision of pseudo labels. Our method is a fully end-to-end single-stage approach that achieves stateof-the-art 71.3% mIoU on PASCAL VOC 2012 and 42.9% mIoU on the MS COCO 2014 dataset, and the proposed TSD-CAM can significantly outperform other singlestage competitors and achieve comparable performance with state-of-the-art multistage methods. Meanwhile, the effectiveness of our method is demonstrated by a large number of ablation experiments, and we provide a new way of thinking to solve the problems of WSSS. Our code is available at: https://github.com/pipizhum/ TSD-CAM.

show abstract

Section: Related Workmentioning

confidence: 99%

TSD-CAM: transformer-based self distillation with CAM similarity for weakly supervised semantic segmentation

Yan,

Chen,

Tang

2024

J. Electron. Imag.

View full text Add to dashboard Cite

show abstract

“…Some methods extract the saliency map (i.e., object localization probability map) from a CNN classifier trained with an image-level annotation [24] [29] [23]. More advanced methods with the same label requirement consist in combining a CNN classifier and, with a saliency map extracted from it, training either another object detection CNN [21] or a semantic segmentation CNN [5], [27]. However, they share in common that the majority of works focus on VOC2007 or VOC2012 datasets where objects are rather centered and occupy a large portion of the image.…”

Section: Weakly Supervised Object Detection In Imagesmentioning

confidence: 99%

Weakly Supervised Faster-RCNN+FPN to classify animals in camera trap images

Pochelu

Erard²,

Cordier

et al. 2022

2022 4th International Conference on Image, Video and Signal Processing

View full text Add to dashboard Cite

Camera traps have revolutionized the animal research of many species that were previously nearly impossible to observe due to their habitat or behavior. They are cameras generally fixed to a tree that take a short sequence of images when triggered. Deep learning has the potential to overcome the workload to automate image classification according to taxon or empty images. However, a standard deep neural network classifier fails because animals often represent a small portion of the high-definition images. That is why we propose a workflow named Weakly Object Detection Faster-RCNN+FPN which suits this challenge. The model is weakly supervised because it requires only the animal taxon label per image but doesn't require any manual bounding box annotations. First, it automatically performs the weakly-supervised bounding box annotation using the motion from multiple frames. Then, it trains a Faster-RCNN+FPN model using this weak supervision. Experimental results have been obtained with two datasets from a Papua New Guinea and Missouri biodiversity monitoring campaign, then on an easily reproducible testbed.CCS Concepts: • Computing methodologies → Neural networks.

show abstract

“…Some methods extract the saliency map (i.e., object localization probability map) from a CNN classifier trained with an image-level annotation [11]. More advanced methods with the same label requirement consist in combining a CNN classifier and, with a saliency map extracted from it, training another semantic segmentation CNN [12]. However, they share in common that the majority of works focus on VOC2007 or VOC2012 datasets where objects are rather centered and occupy a large portion of the image.…”

Section: B Weakly Supervised Object Detection In Images and Videosmentioning

confidence: 99%

Weakly Supervised Faster-RCNN+FPN to classify small animals in camera trap images

Pochelu¹,

Erard²,

Cordier³

et al. 2021

Preprint

View full text Add to dashboard Cite

<div>Camera traps have revolutionized animal research of many species that were previously nearly impossible to observe due to their habitat or behavior.</div><div>Deep learning has the potential to overcome the workload to the class automatically those images according to taxon or empty images. However, a standard deep neural network classifier fails because animals often represent a small portion of the high-definition images. Therefore, we propose a workflow named Weakly Object Detection Faster-RCNN+FPN which suits this challenge. The model is weakly supervised because it requires only the animal taxon label per image but doesn't require any manual bounding box annotations. First, it automatically performs the weakly supervised bounding box annotation using the motion from multiple frames. Then, it trains a Faster-RCNN+FPN model using this weak supervision.<br></div><div>Experimental results have been obtained on two datasets and an easily reproducible testbed.</div>

show abstract

Joint Learning of Saliency Detection and Weakly Supervised Semantic Segmentation

Cited by 68 publications

References 44 publications

TSD-CAM: transformer-based self distillation with CAM similarity for weakly supervised semantic segmentation

TSD-CAM: transformer-based self distillation with CAM similarity for weakly supervised semantic segmentation

Weakly Supervised Faster-RCNN+FPN to classify animals in camera trap images

Weakly Supervised Faster-RCNN+FPN to classify small animals in camera trap images

Contact Info

Product

Resources

About