Consistent visual words mining with adaptive sampling

Abstract

State-of-the-art large-scale object retrieval systems usually combine efficient Bag-of-Words indexing models with a spatial verification re-ranking stage to improve query performance. In this paper we propose to directly discover spatially verified visual words as a batch process. Contrary to previous related methods based on feature sets hashing or clustering, we suggest not trading recall for efficiency by sticking on an accurate two-stage matching strategy. The problem then rather becomes a sampling issue: how to effectively and efficiently select relevant query regions while minimizing the number of tentative probes? We therefore introduce an adaptive weighted sampling scheme, starting with some prior distribution and iteratively converging to unvisited regions. Interestingly, the proposed paradigm is generalizable to any input prior distribution, including specific visual concept detectors or efficient hashing-based methods. We show in the experiments that the proposed method allows to discover highly interpretable visual words while providing excellent recall and image representativity.

Scalable mining of small visual objects

“…The scalable mining of visual content database is recently emerging [2,15,11,18,17,19]. Most of these methods are dedicated and tested to the discovery of large objects (more than 20% of the images) in image collections [2,15,17,19].…”

“…In [2] is presented a qualitative evaluation of small visual objects mining in large scale dataset. Consistent Visual Words Mining with Adaptive Sampling [11] is a rare work where the technology is dedicated to discover and mine small visual objects and where a quantitative evaluation is presented.…”

“…They also proposed another prior to avoid textured areas (to suppress false alarms). In [11], the priors only exploit intra-image saliency information.…”

“…Seeds Verification and Retrieval: In [11], to suppress the ROI containing false alarms where there are no object instances, they directly propose to launch a process of local region search. In fact, the selected seeds or ROI are queries of this process.…”

“…The framework, proposed in [11], is then flexible and generic, because it could be possible to define new types of priors to discover different objects or to change the discovering strategy. The main computational cost of this framework is the last step: Seeds Verification and Retrieval.…”