Deeply Activated Salient Region for Instance Search

Abstract

The performance of instance search depends heavily on the ability to locate and describe a wide variety of object instances in a video/image collection. Due to the lack of proper mechanism in locating instances and deriving feature representation, instance search is generally only effective for retrieving instances of known object categories. In this paper, a simple but effective instance-level feature representation is presented. Different from other approaches, the issues in class-agnostic instance localization and distinctive feature representation are considered. The former is achieved by detecting salient instance regions from an image by a layer-wise back-propagation process. The back-propagation starts from the last convolution layer of a pre-trained CNN that is originally used for classification. The back-propagation proceeds layer-by-layer until it reaches the input layer. This allows the salient instance regions in the input image from both known and unknown categories to be activated. Each activated salient region covers the full or more usually a major range of an instance. The distinctive feature representation is produced by average-pooling on the feature map of certain layer with the detected instance region. Experiments show that such kind of feature representation demonstrates considerably better performance over most of the existing approaches. In addition, we show that the proposed feature descriptor is also suitable for content-based image search.

Combating Ambiguity for Hash-code Learning in Medical Instance Retrieval

“…When encountering a dubious diagnostic case, CBIR systems can help radiologists search for similar cases in their decision-making process. Instance-level image retrieval is to hunt for images with the same instance as a query image in a large image database [3]. The benefit of instance-level retrieval for medical image screening and diagnosing can be witnessed in an observer study [4].…”

Towards Accurate Localization by Instance Search

“…Visual object localization from image/video frame is a crucial step in object detection [29,30], visual object tracking [1,21,22], as well as instance search [23,47,49], etc. It requires to outline the position (usually given as a bounding box) where the target object is located.…”

“…Recent research [47] shows that deeply activated regions that are output by a pre-trained image classification network for an input image coincide well with the instances inside the image. Features derived from these regions demonstrate good performance in instance search task.…”

“…Features derived from these regions demonstrate good performance in instance search task. Additionally, this method is able to detect objects from both the known and unknown categories since these regions are localized by backpropagating from the class-agnostic layer [47]. Unfortunately, only several sub-regions of an object are activated to support the classification.…”

“…In this paper, we aim to achieve accurate object localization on the top-ranked images returned by instance search [47]. Although it sounds like a post-processing step for instance search task, it turns out to be a novel way to address the detection issue in few-shot setting.…”