Unsupervised Cross‐Media Hashing Learning via Knowledge Graph

Abstract

With the rapid growth of multimedia data, cross‐media hashing has become an important technology for fast cross‐media retrieval. Because the manual annotations are difficult to obtain in real‐world application, unsupervised cross‐media hashing is studied to address the hashing learning without manual annotations. Existing unsupervised cross‐media hashing methods generally focus on calculating the similarities through the features of multimedia data, while the learned hashing code cannot reflect the semantic relationship among the multimedia data, which hinders the accuracy in the cross‐media retrieval. When humans try to understand multimedia data, the knowledge of concept relations in our brain plays an important role in obtaining high‐level semantic. Inspired by this, we propose a knowledge guided unsupervised cross‐media hashing (KGUCH) approach, which applies the knowledge graph to construct high‐level semantic correlations for unsupervised cross‐media hash learning. Our contributions in this paper can be summarized as follows: 1) The knowledge graph is introduced as auxiliary knowledge to construct the semantic graph for the concepts in each image and text instance, which can bridge the multimedia data with high‐level semantic correlations to improve the accuracy of learned hash codes for cross‐media retrieval. 2) The proposed KGUCH approach constructs correlation of the multimedia data from both the semantic and the feature aspects, which can exploit complementary information to promote the unsupervised cross‐media hash learning. The experiments are conducted on three widely‐used datasets, which verify the effectiveness of our proposed KGUCH approach.