Self-constraining and attention-based hashing network for bit-scalable cross-modal retrieval

Wang, Xinzhi; Zou, Xitao; Bakker, Erwin M.; Wu, Song

doi:10.1016/j.neucom.2020.03.019

Cited by 62 publications

(29 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our experiments, we utilize mean average precision (MAP), top N -precision curves (top N Curves), and precision-recall curves (PR Curves) as evaluation metrics; for the detailed description of these evaluation metrics, refer to [ 22 , 23 ].…”

Section: Methodsmentioning

confidence: 99%

Quadruplet‐Based Deep Cross‐Modal Hashing

Liu

Xiong

Zhang

et al. 2021

Computational Intelligence and Neuroscience

Self Cite

View full text Add to dashboard Cite

Recently, benefitting from the storage and retrieval efficiency of hashing and the powerful discriminative feature extraction capability of deep neural networks, deep cross-modal hashing retrieval has drawn more and more attention. To preserve the semantic similarities of cross-modal instances during the hash mapping procedure, most existing deep cross-modal hashing methods usually learn deep hashing networks with a pairwise loss or a triplet loss. However, these methods may not fully explore the similarity relation across modalities. To solve this problem, in this paper, we introduce a quadruplet loss into deep cross-modal hashing and propose a quadruplet-based deep cross-modal hashing (termed QDCMH) method. Extensive experiments on two benchmark cross-modal retrieval datasets show that our proposed method achieves state-of-the-art performance and demonstrate the efficiency of the quadruplet loss in cross-modal hashing.

show abstract

Section: Methodsmentioning

confidence: 99%

Quadruplet‐Based Deep Cross‐Modal Hashing

Liu

Xiong

Zhang

et al. 2021

Computational Intelligence and Neuroscience

Self Cite

View full text Add to dashboard Cite

show abstract

“…One of the most typical is deep cross-modal hashing (DCMH) ( Jiang & Li, 2017 ), which firstly applies the deep learning architecture to cross-modal hashing retrieval. The self-constraint and attention-based hashing network (SCAHN) ( Wang et al, 2020a ) explores the hash representations of intermediate layers in an adaptive attention matrix. The correlation hashing network (CHN) ( Cao et al, 2016 ) adopts the triplet loss measured by cosine distance to reveal the semantic relationship between instances and acquires high-ranking hash codes.…”

Section: Related Workmentioning

confidence: 99%

“…Moreover, DNN based models have illustrated great advantages over other hand-crafted shallow models. To name a few, Deep Cross-Modal Hashing (DCMH) ( Jiang & Li, 2017 ), Self-Super Adversarial Hashing (SSAH) Li et al (2018) , Correlation Hashing Network (CHN) ( Cao et al, 2016 ), Self-Constraint and Attention-based Hashing Network (SCAHN) ( Wang et al, 2020a ), Triplet-based Deep Hashing (TDH) ( Deng et al, 2018 ), Self-Constraining and Attention-based Hashing Network (SCAHN) ( Wang et al, 2020b ), Pairwise Relationship Guided Deep Hashing (PRDH) ( Yang et al, 2017 ) and Multi-Label Semantics Preserving Hashing (MLSPH) ( Zou et al, 2021 ). However, these DNN based models still suffer from the following disadvantages.…”

Section: Introductionmentioning

confidence: 99%

Hierarchical semantic interaction-based deep hashing network for cross-modal retrieval

Chen

Wang

2021

PeerJ Computer Science

Self Cite

View full text Add to dashboard Cite

Due to the high efficiency of hashing technology and the high abstraction of deep networks, deep hashing has achieved appealing effectiveness and efficiency for large-scale cross-modal retrieval. However, how to efficiently measure the similarity of fine-grained multi-labels for multi-modal data and thoroughly explore the intermediate layers specific information of networks are still two challenges for high-performance cross-modal hashing retrieval. Thus, in this paper, we propose a novel Hierarchical Semantic Interaction-based Deep Hashing Network (HSIDHN) for large-scale cross-modal retrieval. In the proposed HSIDHN, the multi-scale and fusion operations are first applied to each layer of the network. A Bidirectional Bi-linear Interaction (BBI) policy is then designed to achieve the hierarchical semantic interaction among different layers, such that the capability of hash representations can be enhanced. Moreover, a dual-similarity measurement (“hard” similarity and “soft” similarity) is designed to calculate the semantic similarity of different modality data, aiming to better preserve the semantic correlation of multi-labels. Extensive experiment results on two large-scale public datasets have shown that the performance of our HSIDHN is competitive to state-of-the-art deep cross-modal hashing methods.

show abstract

“…Seven state-of-the-art cross-modal hashing methods are adopted to compare with SAAGHN, including Semantic Correlation Maximization (SCM) [15], Deep Cross-modal Hashing (DCMH) [23], Cross-modal Hamming Hashing (CMHH) [43], Self-supervised Adversarial Hashing (SSAH) [27], Pairwise Relationship Guided Deep Hashing (PRDH) [25], Correlation Hashing Network (CHN) [24], Semantic-preserving Hashing (SePH) [14] and Self-Constraining and Attention-based Hashing Network (SCAHN) [36].…”

Section: Inputmentioning

confidence: 99%

Self-Attention and Adversary Guided Hashing Network for Cross-Modal Retrieval

Chen¹,

Wang²,

Wu³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Recently deep cross-modal hashing networks have received increasing interests due to its superior query efficiency and low storage cost. However, most of existing methods concentrate less on hash representations learning part, which means the semantic information of data cannot be fully used. Furthermore, they may neglect the high-ranking relevance and consistency of hash codes. To solve these problems, we propose a Self-Attention and Adversary Guided Hashing Network (SAAGHN). Specifically, it employs self-attention mechanism in hash representations learning part to extract rich semantic relevance information. Meanwhile, in order to keep invariability of hash codes, adversarial learning is adopted in the hash codes learning part. In addition, to generate higher-ranking hash codes and avoid local minima early, a new batch semi-hard cosine triplet loss and a cosine quantization loss are proposed. Extensive experiments on two benchmark datasets have shown that SAAGHN outperforms other baselines and achieves the state-of-the-art performance.

show abstract

Self-constraining and attention-based hashing network for bit-scalable cross-modal retrieval

Cited by 62 publications

References 14 publications

Quadruplet‐Based Deep Cross‐Modal Hashing

Quadruplet‐Based Deep Cross‐Modal Hashing

Hierarchical semantic interaction-based deep hashing network for cross-modal retrieval

Self-Attention and Adversary Guided Hashing Network for Cross-Modal Retrieval

Contact Info

Product

Resources

About