Rethinking Localization Map: Towards Accurate Object Perception with Self-Enhancement Maps

Zhang, Xiaolin; Wei, Yunchao; Yang, Yi; Wu, Fei

doi:10.48550/arxiv.2006.05220

Cited by 10 publications

(21 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…DANet [43] applied a divergent activation to learn complementary visual cues for WSOL. SEM [50] refined the localization maps by using the point-wise similarity within seed regions. GC-Net [19] took geometric shapes into account and proposed a multi-task loss function for WSOL.…”

Section: Related Workmentioning

confidence: 99%

“…To alleviate such a limitation, one solution is to utilize pixel similarity and global cues to refine activation maps [38,39,49,50]. Cao et al [4] found that the global contexts modeled by non-local networks are almost the same for query positions and thereby proposed NLNet [38] with SENet [12] for global context modeling.…”

Section: Related Workmentioning

confidence: 99%

“…In Fig 7, we compare localization accuracy among variant CAM methods under different IoUs on CUB-200-2011 [35]. TS-CAM outperforms the CAM [51] and RCAM [50] under each IoU by large margins. In addition, TS-CAM achieves larger gains as IoU threshold increases, which indicates that the localization maps of our method cover the object extent accurately.…”

Section: Performancementioning

confidence: 99%

See 2 more Smart Citations

TS-CAM: Token Semantic Coupled Attention Map for Weakly Supervised Object Localization

Gao

Wan

Pan

et al. 2021

2021 IEEE/CVF International Conference on Computer Vision (ICCV)

158

View full text Add to dashboard Cite

Weakly supervised object localization (WSOL) is a challenging problem when given image category labels but requires to learn object localization models. Optimizing a convolutional neural network (CNN) for classification tends to activate local discriminative regions while ignoring complete object extent, causing the partial activation issue. In this paper, we argue that partial activation is caused by the intrinsic characteristics of CNN, where the convolution operations produce local receptive fields and experience difficulty to capture long-range feature dependency among pixels. We introduce the token semantic coupled attention map (TS-CAM) to take full advantage of the self-attention mechanism in visual transformer for long-range dependency extraction. TS-CAM first splits an image into a sequence of patch tokens for spatial embedding, which produce attention maps of long-range visual dependency to avoid partial activation. TS-CAM then re-allocates category-related semantics for patch tokens, enabling each of them to be aware of object categories. TS-CAM finally couples the patch tokens with the semantic-agnostic attention map to achieve semanticaware localization. Experiments on the ILSVRC/CUB-200-2011 datasets show that TS-CAM outperforms its CNN-CAM counterparts by 7.1%/27.1% for WSOL, achieving state-ofthe-art performance.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Performancementioning

confidence: 99%

See 1 more Smart Citation

TS-CAM: Token Semantic Coupled Attention Map for Weakly Supervised Object Localization

Gao

Wan

Pan

et al. 2021

2021 IEEE/CVF International Conference on Computer Vision (ICCV)

158

View full text Add to dashboard Cite

show abstract

“…For a similar reason, [46] train two classifiers, where one covers the feature maps of the other. [49] improves the representation of the classifier by employing quantization. In contrast, our method employs a standard pre-trained classifier and obtains state-of-the-art results out-of-the-box.…”

Section: Discussionmentioning

confidence: 99%

“…The divergent activation approach (DA) [42] aggregates and shares information from different spatial layers of the backbone. A similarity score that contrasts high-activation regions with other regions was proposed by [49].…”

Section: Related Workmentioning

confidence: 99%

Learning a Weight Map for Weakly-Supervised Localization

Shaharabany¹,

Wolf²

2021

Preprint

View full text Add to dashboard Cite

In the weakly supervised localization setting, supervision is given as an image-level label. We propose to employ an image classifier f and to train a generative network g that outputs, given the input image, a per-pixel weight map that indicates the location of the object within the image. Network g is trained by minimizing the discrepancy between the output of the classifier f on the original image and its output given the same image weighted by the output of g. The scheme requires a regularization term that ensures that g does not provide a uniform weight, and an early stopping criterion in order to prevent g from over-segmenting the image. Our results indicate that the method outperforms existing localization methods by a sizable margin on the challenging fine-grained classification datasets, as well as a generic image recognition dataset. Additionally, the obtained weight map is also state-of-the-art in weakly supervised segmentation in fine-grained categorization datasets.

show abstract

Analyzing and Improving the Quality and Fitness for Purpose of OpenStreetMap as Labels in Remote Sensing Applications

Schott,

Zell,

Lautenbach

et al. 2023

Volunteered Geographic Information

View full text Add to dashboard Cite

OpenStreetMap (OSM) is a well-known example of volunteered geographic information. It has evolved to one of the most used geographic databases. As data quality of OSM is heterogeneous both in space and across different thematic domains, data quality assessment is of high importance for potential users of OSM data. As use cases differ with respect to their requirements, it is not data quality per se that is of interest for the user but fitness for purpose. We investigate the fitness for purpose of OSM to derive land-use and land-cover labels for remote sensing-based classification models. Therefore, we evaluated OSM land-use and land-cover information by two approaches: (1) assessment of OSM fitness for purpose for samples in relation to intrinsic data quality indicators at the scale of individual OSM objects and (2) assessment of OSM-derived multi-labels at the scale of remote sensing patches ($$1.22 \times 1.22$$ 1.22 × 1.22 km) in combination with deep learning approaches. The first approach was applied to 1000 randomly selected relevant OSM objects. The quality score for each OSM object in the samples was combined with a large set of intrinsic quality indicators (such as the experience of the mapper, the number of mappers in a region, and the number of edits made to the object) and auxiliary information about the location of the OSM object (such as the continent or the ecozone). Intrinsic indicators were derived by a newly developed tool based on the OSHDB (OpenStreetMap History DataBase). Afterward, supervised and unsupervised shallow learning approaches were used to identify relationships between the indicators and the quality score. Overall, investigated OSM land-use objects were of high quality: both geometry and attribute information were mostly accurate. However, areas without any land-use information in OSM existed even in well-mapped areas such as Germany. The regression analysis at the level of the individual OSM objects revealed associations between intrinsic indicators, but also a strong variability. Even if more experienced mappers tend to produce higher quality and objects which underwent multiple edits tend to be of higher quality, an inexperienced mapper might map a perfect land-use polygon. This result indicates that it is hard to predict data quality of individual land-use objects purely on intrinsic data quality indicators. The second approach employed a label-noise robust deep learning method on remote sensing data with OSM labels. As the quality of the OSM labels was manually assessed beforehand, it was possible to control the amount of noise in the dataset during the experiment. The addition of artificial noise allowed for an even more fine-grained analysis on the effect of noise on prediction quality. The noise-tolerant deep learning method was capable to identify correct multi-labels even for situations with significant levels of noise added. The method was also used to identify areas where input labels were likely wrong. Thereby, it is possible to provide feedback to the OSM community as areas of concern can be flagged.

show abstract

Rethinking Localization Map: Towards Accurate Object Perception with Self-Enhancement Maps

Cited by 10 publications

References 40 publications

TS-CAM: Token Semantic Coupled Attention Map for Weakly Supervised Object Localization

TS-CAM: Token Semantic Coupled Attention Map for Weakly Supervised Object Localization

Learning a Weight Map for Weakly-Supervised Localization

Analyzing and Improving the Quality and Fitness for Purpose of OpenStreetMap as Labels in Remote Sensing Applications

Contact Info

Product

Resources

About