TDCMR: Triplet-Based Deep Cross-Modal Retrieval for Geo-Multimedia Data

Abstract

Mass multimedia data with geographical information (geo-multimedia) are collected and stored on the Internet due to the wide application of location-based services (LBS). How to find the high-level semantic relationship between geo-multimedia data and construct efficient index is crucial for large-scale geo-multimedia retrieval. To combat this challenge, the paper proposes a deep cross-modal hashing framework for geo-multimedia retrieval, termed as Triplet-based Deep Cross-Modal Retrieval (TDCMR), which utilizes deep neural network and an enhanced triplet constraint to capture high-level semantics. Besides, a novel hybrid index, called TH-Quadtree, is developed by combining cross-modal binary hash codes and quadtree to support high-performance search. Extensive experiments are conducted on three common used benchmarks, and the results show the superior performance of the proposed method.

Deep Label Feature Fusion Hashing for Cross-Modal Retrieval

“…Deep multiscale fusion hashing for cross-modal retrieval (DMFH) [31] extracted convolution features at different scales for each image data to represent the image data more accurately. Triplet-based deep cross-modal retrieval for geo-multimedia data (TDCMR) [32] applies the improved triplet constraint to generate more accurate hash codes. Semantics-preserving hashing based on multi-scale fusion for cross-modal retrieval (SPHMF) [33] constructs pairwise loss and inter-modal loss of tag generation network to guide hash code learning.…”