Instance Search via Instance Level Segmentation and Feature Representation

Abstract

Instance search is an interesting task as well as a challenging issue due to the lack of effective feature representation. In this paper, an instance level feature representation built upon fully convolutional instance-aware segmentation is proposed. The feature is ROI-pooled from the segmented instance region. So that instances in various sizes and layouts are represented by deep features in uniform length. This representation is further enhanced by the use of deformable ResNeXt blocks. Superior performance is observed in terms of its distinctiveness and scalability on a challenging evaluation dataset built by ourselves. In addition, the proposed enhancement on the network structure also shows superior performance on the instance segmentation task.

Combating Ambiguity for Hash-code Learning in Medical Instance Retrieval

“…The relevancy of instance-level retrieval is mainly grounded on the visual similarity of instances rather than the whole image [5], so the features of a region-wise instance residing in a retrieved image should be explored effectively. Recently, many existing works on instance-level retrieval typically extracted visual features by using convolutional neural networks (CNN) to prevent the visual features unique to an instance from drowning in the global image.…”

Deeply Activated Salient Region for Instance Search

“…Recently, due to the great success of convolutional neural networks (CNNs) in learning high-level semantic features for image classification [7], object detection [8] and instance segmentation [9], [10], CNNs have been introduced to instance search [11]. Using Fast R-CNN [12] as example, the instancewise vector representation is produced through RoI-pooling from the region of feature maps corresponding to a candidate object bounding box.…”

“…The feature captures textureless region and is relatively robust to object deformation, if compared with global and local features. Despite satisfactory performance in instance search as reported in [11], the main drawback of CNN-based solutions is their stringent demand on training data. In [11], for example, pixel-wise annotation of instance location is required.…”

“…Despite satisfactory performance in instance search as reported in [11], the main drawback of CNN-based solutions is their stringent demand on training data. In [11], for example, pixel-wise annotation of instance location is required. The annotation effort is cost expensive and labor intensive.…”

“…Furthermore, the learning process makes the CNNs more sensitive to object instances of known categories, by treating unseen categories as background class [9]. As a result, approach in [11] is only able to deal effectively with instances belonging to known object categories. This problem remains unaddressed if one switches to relying on CNN-based object detection framework [8], [13], [14] for instance-level feature extraction, such as [15].…”