Selective Convolutional Descriptor Aggregation for Fine-Grained Image Retrieval

Abstract

Deep convolutional neural network models pre-trained for the ImageNet classification task have been successfully adopted to tasks in other domains, such as texture description and object proposal generation, but these tasks require annotations for images in the new domain. In this paper, we focus on a novel and challenging task in the pure unsupervised setting: fine-grained image retrieval. Even with image labels, fine-grained images are difficult to classify, letting alone the unsupervised retrieval task. We propose the selective convolutional descriptor aggregation (SCDA) method. The SCDA first localizes the main object in fine-grained images, a step that discards the noisy background and keeps useful deep descriptors. The selected descriptors are then aggregated and the dimensionality is reduced into a short feature vector using the best practices we found. The SCDA is unsupervised, using no image label or bounding box annotation. Experiments on six fine-grained data sets confirm the effectiveness of the SCDA for fine-grained image retrieval. Besides, visualization of the SCDA features shows that they correspond to visual attributes (even subtle ones), which might explain SCDA's high-mean average precision in fine-grained retrieval. Moreover, on general image retrieval data sets, the SCDA achieves comparable retrieval results with the state-of-the-art general image retrieval approaches.

Component-Based Attention for Large-Scale Trademark Retrieval

“…There exist two kinds of attention approaches: weighting [3,10] and selection [8,28]. Weighting approaches create attention by emphasizing convolutional activations of relevant information or by reducing activation of irrelevant information via multiplying weights.…”

“…Selection approaches direct attention to import information by selecting convolutional features; and the process is equivalent to applying a binary weight spatially in the case of using global average pooling or maximum activation pooling as aggregation techniques. For example, Wei et al [28] selected local features on the largest activated connected component of a convolutional layer. Hoang et al [8] select deep convolutional local features via masks (i.e.…”

Scene Recognition With Prototype-Agnostic Scene Layout

“…Zheng et al [24] group the convolutional channels to localize object parts in the well constrained spatial configurations. Wei et al [25] use a simple thresholding method to discover object parts and select the largest component to represent the desired foreground object. In contrast, we formulate the discovery procedure for scene recognition, where more complex semantic regions and unconstrained spatial structures exist.…”

Image classification and captioning model considering a CAM‐based disagreement loss

“…Most fine-grained classification systems employ visual features of images to classify objects using a CNN [25][26][27][28][29], and subordinate classes from various domains such as flowers, birds, dogs, aircrafts, and cars can be successfully recognized using these approaches. The objects are visually similar to each other, and can only be discriminated through subtle details.…”

“…The objects are visually similar to each other, and can only be discriminated through subtle details. Most fine-grained classification systems employ visual features of images to classify objects using a CNN [25][26][27][28][29], and subordinate classes from various domains such as flowers, birds, dogs, aircrafts, and cars can be successfully recognized using these approaches. To improve the classification performance, some approaches employ hierarchical semantic information such as a taxonomic rank [30], the semantic distance of WordNet [31], and text [15,17].…”