Anchor Graph Structure Fusion Hashing for Cross-Modal Similarity Search

Abstract

Cross-modal hashing has been widely applied to retrieve items across modalities due to its superiority in fast computation and low storage. However, some challenges are still needed to address: (1) most existing CMH methods take graphs, which are always predefined separately in each modality, as input to model data distribution. These methods omit to consider the correlation of graph structure among multiple modalities. Besides, cross-modal retrieval results highly rely on the quality of predefined affinity graphs; (2) most existing CMH methods deal with the preservation of intra-and inter-modal affinity independently to learn the binary codes, which ignores considering the fusion affinity among multi-modalities data;(3) most existing CMH methods relax the discrete constraints to solve the optimization objective, which could significantly degrade the retrieval performance. To solve the above limitations, in this paper, we propose a novel Anchor Graph Structure Fusion Hashing (AGSFH). AGSFH constructs the anchor graph structure fusion matrix from different anchor graphs of multiple modalities with the Hadamard product, which can fully exploit the geometric property of underlying data structure across multiple modalities. Specifically, based on the anchor graph structure fusion matrix, AGSFH makes an attempt to directly learn an intrinsic anchor graph, where the structure of the intrinsic anchor graph is adaptively tuned so that the number of components of the intrinsic graph is exactly equal to the number of clusters. Based on this process, training instances can be clustered into semantic space. Besides, AGSFH preserves the anchor fusion affinity into the common binary Hamming space, capturing intrinsic similarity and structure across modalities by hash codes. Furthermore, a discrete optimization framework is designed to learn the unified binary codes across modalities. Extensive experimental results on three public social datasets demonstrate the superiority of AGSFH in cross-modal retrieval.

Supervised Consensus Anchor Graph Hashing for Cross Modal Retrieval

“…[36] constructs multi-view affinity and asymmetric graphs over anchor data which serve as a unified semantic hub in a semi-supervised manner. Anchor graph structure fusion hashing [40] incorporates intrinsic anchor fusion affinity preservation and clustering structure optimization into a unified framework. Asymmetric Transfer Hashing [41] characterizes the domain distribution gap by minimizing two asymmetric hash functions and learns an adaptive bipartite graph to characterize the similarity between cross-domain samples.…”