Query Bootstrapping: A Visual Mining Based Query Expansion

Abstract

SUMMARYBag of Visual Words (BoVW) is an effective framework for image retrieval. Query expansion (QE) further boosts retrieval performance by refining a query with relevant visual words found from the geometric consistency check between the query image and highly ranked retrieved images obtained from the first round of retrieval. Since QE checks the pairwise consistency between query and highly ranked images, its performance may deteriorate when there are slight degradations in the query image. We propose Query Bootstrapping as a variant of QE to circumvent this problem by using the consistency of highly ranked images instead of pairwise consistency. In so doing, we regard frequently co-occurring visual words in highly ranked images as relevant visual words. Frequent itemset mining (FIM) is used to find such visual words efficiently. However, the FIM-based approach requires sensitive parameters to be fine-tuned, namely, support (min/max-support) and the number of top ranked images (top-k). Here, we propose an adaptive support algorithm that adaptively determines both the minimum support and maximum support by referring to the first round's retrieval list. Selecting relevant images by using a geometric consistency check further boosts retrieval performance by reducing outlier images from a mining process. An important parameter for the LO-RANSAC algorithm that is used for the geometric consistency check, namely, inlier threshold, is automatically determined by our algorithm. We further introduce tf-fi-idf on top of tf-idf in order to take into account the frequency of inliers (fi) in the retrieved images. We evaluated the performance of QB in terms of mean average precision (mAP) on three benchmark datasets and found that it gave significant performance boosts of 5.37%, 9.65%, and 8.52% over that of state-of-the-art QE on Oxford 5k, Oxford 105k, and Paris 6k, respectively. key words: image retrieval, instance search, query expansion, frequent itemset mining, visual word mining, query bootstrapping, adaptive support, adaptive inlier threshold

A Vehicle Reidentification Algorithm Based on Double-Channel Symmetrical CNN

“…e above changes will seriously affect the performance of vehicle recognition. At present, the research on target reidentification mainly focuses on the field of pedestrian reidentification [7][8][9][10] and is rarely applied to other targets. Since 2015, a small number of scholars have tried to enter the field of vehicle reidentification, but they can only be applied to images of the same scale and angle, with weak robustness to environmental changes or based on small datasets.…”

Discovering informative features in large-scale landmark image collection

“…This threshold is called the minimum support (min-sup for short). We apply the adaptive support method introduced by Kasamwattanarote et al [9], where the produced frequent patterns fall between the minimum and maximum support thresholds that produce the maximum number of patterns using FIM in 'maximal' frequent itemsets mode. Next, FIM runs in the 'closed' itemsets mode with the optimal min-sup and max-sup to generate actual patterns.…”

“…FIM has been successfully integrated with the computer vision applications, where it can be applied to extract the frequent co-occurring feature as the object frequently appears among multiple images or even on video frames. This method has been employed in several applications such as video mining [5], visual phrase mining [6], mining multiple queries [7], re-ranking and classification [8] and query expansion [9].…”