Unveiling the Potential of Structure Preserving for Weakly Supervised Object Localization

Abstract

Weakly supervised object localization (WSOL) remains an open problem given the deficiency of finding object extent information using a classification network. Although prior works struggled to localize objects through various spatial regularization strategies, we argue that how to extract object structural information from the trained classification network is neglected. In this paper, we propose a two-stage approach, termed structure-preserving activation (SPA), toward fully leveraging the structure information incorporated in convolutional features for WSOL. First, a restricted activation module (RAM) is designed to alleviate the structuremissing issue caused by the classification network on the basis of the observation that the unbounded classification map and global average pooling layer drive the network to focus only on object parts. Second, we designed a post-process approach, termed self-correlation map generating (SCG) module to obtain structure-preserving localization maps on the basis of the activation maps acquired from the first stage. Specifically, we utilize the high-order self-correlation (HSC) to extract the inherent structural information retained in the learned model and then aggregate HSC of multiple points for precise object localization. Extensive experiments on two publicly available benchmarks including CUB-200-2011 and ILSVRC show that the proposed SPA achieves substantial and consistent performance gains compared with baseline approaches. Code and models are available at github.com/Panxjia/SPA CVPR2021.

Distributed Attention for Grounded Image Captioning

“…Contextual information [19], attention mechanism [43], gradient map [40] and semantic segmentation [48] are leveraged to learn accurate object proposals. CAM-based methods [33,41,41,55,57] produce localization maps by aggregating deep feature maps using a class-specific fully connected layer. Despite the simplicity and effectiveness of CAM-based methods, they suffer from identifying small discriminative parts of objects.…”

“…SPG [56] and I 2 C [57] increased the quality of localization maps by introducing the constraint of pixel-level correlations into the network. SPA [33] and TS-CAM [13] obtain accurate localization maps with the help of long-range structural information. Different from the WSOD task, our task leverages the correctly attended regions for caption generation.…”