SIFT Feature-Based Second-Order Image Hash Retrieval Approach

Zhao, Xianghui; Li, Zhirong; Yi, Junkai

doi:10.17706/jsw.13.2.103-116

Cited by 10 publications

(8 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Aiming to obtain a comprehensive experimental analysis of our MMFR method, we conduct eight comparison methods which are SCRL [19], DTBH [18], DIVR [17], CMIR [16], DVAN [15], CNN + SPEC [64], DBLP [65], and SIFT + M [66]. The SCRL [19] is a method which considers the pairwise consistency, intra-modality consistency and Non-paired intermodality consistency in RSIAR.…”

Section: F Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Multimodal Fusion Remote Sensing Image–Audio Retrieval

Yang

Wang

Sun

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Remote sensing image-audio retrieval (RSIAR) has been an emerging research topic in recent years, and many different methods have been proposed for this topic. These RSIAR methods have achieved good retrieval results, but two problems remain: the lack of discriminability of audio modality and the existence of a heterogeneous gap between audio and image. These two problems make the cross-modal common embedding space for audio and images suboptimal, often failing to perform superior retrieval. This paper proposes a novel RSIAR method named multi-modal fusion remote sensing imageaudio retrieval (MMFR) to address these two problems. MMFR first converts original audio input to text. Then MMFR uses a feature fusion module to obtain a fusion representation fused with text information instead of the original sole audio representation. Fusion text information can make the pronunciationbased audio feature more semantically discriminable and convert pronunciation-based audio feature to more "high level" fusion feature to cross the heterogeneous gap. Seven different fusion methods are tried in the feature fusion module. In addition, the triplet loss, the semantic loss and the consistency loss are used to optimize the common retrieval space. Extensive experiments conducted on the UCM IV, RSICD IV, and SYDNE IV datasets demonstrate that our MMFR method outperforms state-of-theart methods.

show abstract

Section: F Resultsmentioning

confidence: 99%

“…The DBLP [65] is another unsupervised method outputting a similarity score between image and speech caption. The SIFT + M [66] uses the sift features of images and the MFCC features of audio to achieve cross-modal retrieval.…”

Section: F Resultsmentioning

confidence: 99%

Multimodal Fusion Remote Sensing Image–Audio Retrieval

Yang

Wang

Sun

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

show abstract

“…In this subsection, nine current state-of-the-art cross-modal retrieval methods are used, including: SIFT+M [42], CMFH [43], DBLP [44], CNN [45], DVAN [18], CMIR-NET [35], DIVR [46], SePHklr [47], and DTBH [34]. The experiment was implemented on three benchmark datasets, which are Sydney Image-Voice Dataset, UCM Image-Voice Dataset, and RSICD Image-Voice Dataset.…”

Section: E Comparison Results On Benchmark Datasetsmentioning

confidence: 99%

Remote Sensing Cross-Modal Retrieval by Deep Image-Voice Hashing

Zhang

Zheng

2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Remote sensing image retrieval aims at searching remote sensing images of interest among immense volumes of remote sensing data, which is an enormous challenge. Direct use of voice for human-computer interaction is more convenient and intelligent. In this paper, a Deep Image-Voice Hashing (DIVH) method is proposed for remote sensing image-voice retrieval. First, the whole framework is composed of the image and the voice feature learning subnetwork. Then, the hash code learning procedure will be leveraged in remote sensing imagevoice retrieval to further improve retrieval efficiency and reduce memory footprint. Hash code learning maps the deep features of images and voices into a common Hamming space. Finally, imagevoice pairwise loss is proposed, which considers the similarity preservation and balance of hash codes. The similarity preserving and the balance controlling term of the loss function improve the similarity preservation from original data space to Hamming space and the discrimination of binary code, respectively. This unified cross-modal feature and hash code learning framework significantly reduce the semantic gap between the two modal data. Experiments demonstrate that the proposed DIVH method can achieve superior retrieval performance than other state-ofthe-art remote sensing image-voice retrieval methods.

show abstract

“…In order to assess the effectiveness of the proposed SCRL method, 7 comparison methods are adopted, including SIFT+M [53], DBLP [54], CNN+SPEC [55], DVAN [11], CMIR-NET [8], DIVR [14], and DTBH [13] methods. The SIFT+M method [53] leverages SIFT features of images and MFCC features of voices to perform the RS image-voice retrieval. The DBLP method [54] adopts an unsupervised manner to learn the coherence between audio and visual modalities.…”

Section: Evaluation Metrics and Comparison Methodsmentioning

confidence: 99%

Semantics-Consistent Representation Learning for Remote Sensing Image-Voice Retrieval

Ning,

Zhao,

Yuan

2021

Preprint

View full text Add to dashboard Cite

With the development of earth observation technology, massive amounts of remote sensing (RS) images are acquired. To find useful information from these images, cross-modal RS image-voice retrieval provides a new insight. This paper aims to study the task of RS image-voice retrieval so as to search effective information from massive amounts of RS data. Existing methods for RS image-voice retrieval rely primarily on the pairwise relationship to narrow the heterogeneous semantic gap between images and voices. However, apart from the pairwise relationship included in the datasets, the intra-modality and non-paired inter-modality relationships should also be taken into account simultaneously, since the semantic consistency among non-paired representations plays an important role in the RS image-voice retrieval task. Inspired by this, a semantics-consistent representation learning (SCRL) method is proposed for RS imagevoice retrieval. The main novelty is that the proposed method takes the pairwise, intra-modality, and non-paired inter-modality relationships into account simultaneously, thereby improving the semantic consistency of the learned representations for the RS image-voice retrieval. The proposed SCRL method consists of two main steps: 1) semantics encoding and 2) semantics-consistent representation learning. Firstly, an image encoding network is adopted to extract high-level image features with a transfer learning strategy, and a voice encoding network with dilated convolution is devised to obtain high-level voice features. Secondly, a consistent representation space is conducted by modeling the three kinds of relationships to narrow the heterogeneous semantic gap and learn semantics-consistent representations across two modalities. Extensive experimental results on three challenging RS image-voice datasets, including Sydney, UCM and RSICD image-voice datasets, show the effectiveness of the proposed method.

show abstract

SIFT Feature-Based Second-Order Image Hash Retrieval Approach

Cited by 10 publications

References 11 publications

Multimodal Fusion Remote Sensing Image–Audio Retrieval

Multimodal Fusion Remote Sensing Image–Audio Retrieval

Remote Sensing Cross-Modal Retrieval by Deep Image-Voice Hashing

Semantics-Consistent Representation Learning for Remote Sensing Image-Voice Retrieval

Contact Info

Product

Resources

About