G-CAM: Graph Convolution Network Based Class Activation Mapping for Multi-label Image Recognition

Abstract

In most multi-label image recognition tasks, human visual perception keeps consistent for different spatial transforms of the same image. Existing approaches either learn the perceptual consistency with only image-level supervision or preserve the middle-level feature consistency of attention regions but neglect the (global) label dependencies between different objects over the dataset. To address this issue, we integrate graph convolution network (GCN) and propose G-CAM, which learns visual attention consistency via GCN based class attention mapping (CAM) for multi-label image recognition. G-CAM consists of an image feature extraction module to generate the feature maps of the original image and its transformed one and a GCN module to learn weighted classifiers that capture the label dependencies between different objects. Different from previous works which use fully-connected classification layer, G-CAM first fuses weighted classifiers with the feature vector to generate the predicted labels for each input image, then combines weighted classifiers with the feature maps to respectively obtain the transformed attention heatmaps of the original image and the attention heatmaps of its transformed one. We can compute the attention consistency loss according to the distance between these two attention heatmaps. Finally, this loss is combined with the multi-label classification loss to update the whole network in an end-to-end manner. We conduct extensive experiments on three multi-label image datasets including FLICKR25K, MS-COCO and NUS-WIDE. Experimental results demonstrate G-CAM can achieve better performance compared with the state-of-the-art multi-label image recognition methods. CCS CONCEPTS• Computing methodologies → Image representations.

MSFA: Multi‐stage feature aggregation network for multi‐label image recognition

“…Despite saving a lot of extra computation required for precise positioning, the process of clipping local regions of the original image and repeating the feature extraction and classification is still not intuitively optimal. Wang et al [13] generated the class‐aware heatmaps via class activation mapping (CAM) and tested the consistency of visual attention for each input sample under different transforms to ensure that the network was making judgments according to the features of the category‐related regions. More granular, SST [9] utilized Transformer [24] to capture features' long‐range dependencies, which facilitate the feature aggregation of objects from the same category.…”

“…Hence, another avenue for improvement lies in related researches that optimize multi-label classification by identifying category-related regions and establishing connections between category-related features and labels. These methods [9][10][11][12][13][14][15][16][17][18][19] encourage the network to extract semantically consistent discriminative features for distinguishing different categories based on category-related local regions, which is crucial in multi-label image classification. However, they tend to neglect the potential impact of multi-scale variations of targets in multi-label image classification scenarios.…”

A Novel Framework for Semi-supervised Multiple-label Image Classification using Multi-stage CNN and Visual Attention Mechanism

“…They are unable to finish end-to-end training because they update the entire network using a subpar multistep training workflow. The study [20] suggests employing www.ijacsa.thesai.org GCAM to record the label relationships between diverse image transforms. Graph convolution network (GCN) is used by MLGCN [11] and AGCN [12] to produce label cooccurrence embedding for multiple label picture categorizations.…”