Remote Sensing Cross-Modal Retrieval by Deep Image-Voice Hashing

Abstract

Remote sensing image retrieval aims at searching remote sensing images of interest among immense volumes of remote sensing data, which is an enormous challenge. Direct use of voice for human-computer interaction is more convenient and intelligent. In this paper, a Deep Image-Voice Hashing (DIVH) method is proposed for remote sensing image-voice retrieval. First, the whole framework is composed of the image and the voice feature learning subnetwork. Then, the hash code learning procedure will be leveraged in remote sensing imagevoice retrieval to further improve retrieval efficiency and reduce memory footprint. Hash code learning maps the deep features of images and voices into a common Hamming space. Finally, imagevoice pairwise loss is proposed, which considers the similarity preservation and balance of hash codes. The similarity preserving and the balance controlling term of the loss function improve the similarity preservation from original data space to Hamming space and the discrimination of binary code, respectively. This unified cross-modal feature and hash code learning framework significantly reduce the semantic gap between the two modal data. Experiments demonstrate that the proposed DIVH method can achieve superior retrieval performance than other state-ofthe-art remote sensing image-voice retrieval methods.

Attention-based localized hash retrieval for large-scale remote sensing images using deep feature splitting strategy

“…The results show that the proposed model achieves competing performance with DHCNN 25 for the UCM and AID datasets. The PatternNet dataset shows a slight increase in performance than the proposed model in DAH 33 . However, by employing the DFS-LHash strategy, we achieve better results than some deep hashing approaches listed in Table 5.…”

“…Table 5 lists the mAP values obtained using different hashing methods like distance attention hashing (DAH), hashing net (HNET), greedy hash (GHASH), deep hashing neural network (DHNN), deep hashing using CNN (DHCNN), and deep contrastive self-supervised hashing (DCSSH), presented in Refs. 25, 33, 40, 44–46.…”

“…A multiview graph convolutional hashing method has been proposed for multisource RSI retrieval 32 . A deep attention hashing with distance-adaptive ranking is presented 33 . A fine-aligned discriminative hashing is designed for cross-modal remote sensing image-audio retrieval 34 .…”

Proxy-Based Rotation Invariant Deep Metric Learning for Remote Sensing Image Retrieval

“…Chen et al [24] propose a new Deep Significance Smoothed Hashing (DSSH) algorithm to focus on the local fine-grained features and saliency information for drone images. Zhao et al [25] obtain finer-grained multi-scale features and achieve a larger receptive field by incorporating the proposed multiscale residual blocks, and the proposed multicontext attention modules increase the perceptual field by aggregating contextual information along channels and spatial dimensions. Experimental results show that this method achieves excellent results.…”

Remote Sensing Image Retrieval by Deep Attention Hashing With Distance-Adaptive Ranking

“…Bao and Guo [16] implemented comparative experiments of eight different similarity measurements for remote sensing image retrieval. The hashing method retrieves the image with the lowest Hamming distance relative to the query image, which takes linear time [17]. The differences between features are not only reflected in the differences in the values of the feature vectors, but also in the overall category differences.…”