Semantic-aware framework for Mobile Image Search

Abstract

Social and Mobile are the two very characterizing trends of the Internet. Subsequently, the volume of photos with rich social, textual and contextual information increases exponentially either on mobile devices or social networks. Performing an efficient and effective mobile Image Search over social photo collection is therefore a crucial challenge. Indeed, capture the complex connections among social photos is as important as speeding up similarity search at large scale. This paper present a generic Mobile Image Search framework with hypergraph hashing. On the mobile side, users are enabled to formulate whether visual, textual or vocal queries. On the server side, we start by modeling complex connections that may exist among photos and social features using an hypergraph. To accelerate the nearest neighbor search over the hypergraph, a spectral hashing is performed. Namely, each hypergraph vertex is mapped to a binary string without loss of similarity. For unseen items in the hypergraph, a query-adaptive supervised learning is carried out to learn binary strings based on the query type. We report the initial results over NUS-WIDE collection which show that the proposed framework is promising in the field of Mobile Image Search.

Visual Re-Ranking via Adaptive Collaborative Hypergraph Learning for Image Retrieval

“…Although promising results are achieved, how to represent complex and high-order relationships hidden in data still the performance bottleneck for graph-based re-ranking. As a generalization of the graph learning, the hypergraph learning is receiving increasing attention in recent years owing to its ability in modeling complex data structure in a more flexible and elegant way [3,23]. Considering the visual re-ranking, the hypergraph learning is widely used for relevance estimation.…”