F-CAM: Full Resolution Class Activation Maps via Guided Parametric Upscaling

Belharbi, Soufiane; Sarraf, Aydin; Pedersoli, Marco; Ayed, Ismail Ben; McCaffrey, Luke; Granger, Éric

doi:10.1109/wacv51458.2022.00378

Cited by 23 publications

(28 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several CAMs variants have been proposed in the literature to address known limitations [ 71 , 72 , 73 ], yield better visualizations, and obtain finer localization. While some of them rely on the existing network structure [ 71 , 74 , 75 , 76 , 77 , 78 ], other techniques require the use of ad hoc architectures [ 73 , 79 , 80 , 81 ]. In this work, four state-of-the-art techniques will be analyzed for the generation of bounding boxes: CAM [ 37 ], Grad-CAM [ 74 ], Grad-CAM++ [ 75 ] and Smooth Grad-CAM++ [ 71 ].…”

Section: Methodsmentioning

confidence: 99%

Proposals Generation for Weakly Supervised Object Detection in Artwork Images

2022

View full text Add to dashboard Cite

Object Detection requires many precise annotations, which are available for natural images but not for many non-natural data sets such as artworks data sets. A solution is using Weakly Supervised Object Detection (WSOD) techniques that learn accurate object localization from image-level labels. Studies have demonstrated that state-of-the-art end-to-end architectures may not be suitable for domains in which images or classes sensibly differ from those used to pre-train networks. This paper presents a novel two-stage Weakly Supervised Object Detection approach for obtaining accurate bounding boxes on non-natural data sets. The proposed method exploits existing classification knowledge to generate pseudo-ground truth bounding boxes from Class Activation Maps (CAMs). The automatically generated annotations are used to train a robust Faster R-CNN object detector. Quantitative and qualitative analysis shows that bounding boxes generated from CAMs can compensate for the lack of manually annotated ground truth (GT) and that an object detector, trained with such pseudo-GT, surpasses end-to-end WSOD state-of-the-art methods on ArtDL 2.0 (≈41.5% mAP) and IconArt (≈17% mAP), two artworks data sets. The proposed solution is a step towards the computer-aided study of non-natural images and opens the way to more advanced tasks, e.g., automatic artwork image captioning for digital archive applications.

show abstract

Section: Methodsmentioning

confidence: 99%

Proposals Generation for Weakly Supervised Object Detection in Artwork Images

2022

View full text Add to dashboard Cite

show abstract

“…Over background regions, classifier is constrained to be the most uncertain in classification due to the lack of positive evidence for the corresponding class. In [6], authors use positive and negative evidence in addition to size priors and conditional random field (CRF) [70]. However, fully negative samples were not explored.…”

Section: Related Workmentioning

confidence: 99%

“…In segmentation tasks, weak supervisory signals come under different forms making it more attractive to several real scenarios. This includes scribbles [42,69], points [2], bounding boxes [13,38,35], global image statistics such as the target-region size [34,1,29,33], and image-level labels [6,39,47,71,76]. In this work, we are interested in image-class labels where, for each image, we are given the image class only, such as cancerous or not.…”

Section: Introductionmentioning

confidence: 99%

Negative Evidence Matters in Interpretable Histology Image Classification

Belharbi¹,

Pedersoli²,

Ayed³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Using only global annotations such as the image class labels, weakly-supervised learning methods allow CNN classifiers to jointly classify an image, and yield the regions of interest associated with the predicted class. However, without any guidance at the pixel level, such methods may yield inaccurate regions. This problem is known to be more challenging with histology images than with natural ones, since objects are less salient, structures have more variations, and foreground and background regions have stronger similarities. Therefore, methods in computer vision literature for visual interpretation of CNNs may not directly apply. In this work, we propose a simple yet efficient method based on a composite loss function that leverages information from the fully negative samples, i.e. samples that do not contain positive parts. Our new loss function contains two complementary terms: the first exploits positive evidence collected from the CNN classifier, while the second leverages the fully negative samples from the training dataset. In particular, we equip a pre-trained classifier with a decoder that allows refining the regions of interest. The same classifier is exploited to collect both the positive and negative evidence at the pixel level to train the decoder. This enables to take advantages of the fully negative samples that occurs naturally in the data, without any additional supervision signals and using only the image class as supervision. Compared to several recent related methods, over the public benchmark GlaS for colon cancer and a Camelyon16 patch-based benchmark for breast cancer using three different backbones, we show the substantial improvements introduced by our method. Our results shows the benefits of using both negative and positive evidence, i.e., the one obtained from a classifier and the one naturally available in datasets. We provide an ablation study of both terms. Our code is publicly available 1 .

show abstract

“…Regressor-free WSOL methods. Following Class Activation Mapping (CAM) [38], most approaches obtain the class prediction and the localization map with a classification network only [2,6,22,27,35,37]. To tackle the incomplete localization problem of CAM, several methods adopted an iterative erasing strategy on input images [22,32] or feature maps [27,35] to force the network's attention to the remaining parts of an object.…”

Section: Weakly Supervised Object Localizationmentioning

confidence: 99%

CREAM: Weakly Supervised Object Localization via Class RE-Activation Mapping

Xu¹,

Hou²,

Zhang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Weakly Supervised Object Localization (WSOL) aims to localize objects with image-level supervision. Existing works mainly rely on Class Activation Mapping (CAM) derived from a classification model. However, CAM-based methods usually focus on the most discriminative parts of an object (i.e., incomplete localization problem). In this paper, we empirically prove that this problem is associated with the mixup of the activation values between less discriminative foreground regions and the background. To address it, we propose Class RE-Activation Mapping (CREAM), a novel clustering-based approach to boost the activation values of the integral object regions. To this end, we introduce class-specific foreground and background context embeddings as cluster centroids. A CAM-guided momentum preservation strategy is developed to learn the context embeddings during training. At the inference stage, the re-activation mapping is formulated as a parameter estimation problem under Gaussian Mixture Model, which can be solved by deriving an unsupervised Expectation-Maximization based soft-clustering algorithm. By simply integrating CREAM into various WSOL approaches, our method significantly improves their performance. CREAM achieves the state-of-the-art performance on CUB, ILSVRC and OpenImages benchmark datasets. Code will be available at https://github.com/Jazzcharles/CREAM.

show abstract

F-CAM: Full Resolution Class Activation Maps via Guided Parametric Upscaling

Cited by 23 publications

References 33 publications

Proposals Generation for Weakly Supervised Object Detection in Artwork Images

Proposals Generation for Weakly Supervised Object Detection in Artwork Images

Negative Evidence Matters in Interpretable Histology Image Classification

CREAM: Weakly Supervised Object Localization via Class RE-Activation Mapping

Contact Info

Product

Resources

About