Deep hashing with self-supervised asymmetric semantic excavation and margin-scalable constraint

Yu, Zhengyang; Wu, Song; Dou, Zhihao; Bakker, Erwin M.

doi:10.1016/j.neucom.2022.01.082

Cited by 5 publications

(3 citation statements)

References 70 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most available cross-modal hashing frameworks typically employ semantic labels to simply define the semantic similarity (i.e., similar or dissimilar), which fails to explore the fine-grained semantic relation between two samples [41]. Besides, existing deep hashing usually overlooks some valuable semantic information about human-annotated labels, leading to sub-optimal binary codes [28], [49].…”

Section: Methodsmentioning

confidence: 99%

“…The choice of the margin parameter in most pair-wise or triplet-wise losses relies heavily on hand-tuning. Previous research has shown that the choice of margins can have a remarkably large impact on retrieval performance, which indicates that a well-chosen margin can be valuable for hash function learning [41]. Therefore, a margin scalable loss with the semantic dictionary is proposed to explore the margin value adaptively according to the semantic similarity.…”

Section: Asymmetric Margin-scalable Lossmentioning

confidence: 99%

“…Although deep hashing methods have made great progress in cross-modal retrieval, they still have room for improvement. By counting the number of shared semantic labels, most existing deep cross-modal hashing methods simply define the semantic similarity between heterogeneous modalities, which hardly applies to multi-label cross-modal retrieval [41]. Specifically, if two samples from different modalities share at least one semantic label, they are similar, in which their similarity scores are assigned to '1', otherwise they are dissimilar, which are identified as '0', thereby failing to explore fine-grained multi-label semantic relations.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Deep Self-Supervised Hashing With Fine-Grained Similarity Mining for Cross-Modal Retrieval

Han,

Wang,

Chen

et al. 2024

IEEE Access

View full text Add to dashboard Cite

With the efficiency of storage and retrieval speed, the hashing methods have attracted a lot of attention for cross-modal retrieval applications. In contrast to traditional cross-modal hashing by using handcrafted features, deep cross-modal hashing integrates the advantages of deep learning and hashing methods to encode raw multimodal data into compact binary codes with semantic information preserved. Generally speaking, most of the existing deep cross-modal hashing methods simply define the semantic similarity between heterogeneous modalities by counting the number of shared semantic labels (such as, two samples share at least one label, they are similar, otherwise they are dissimilar), which fails to represent the accurate multi-label semantic relations between heterogeneous data. In this paper, we propose a new Deep Self-supervised Hashing with Fine-grained Similarity Mining (DSH-FSM) framework to efficiently preserve the fine-grained multi-label semantic similarity, learning a highly separable embedding space. Specifically, by employing an asymmetric guidance strategy, a novel Semantic-Network is introduced into cross-modal hashing to learn two semantic dictionaries, including the semantic feature dictionary and the semantic code dictionary, which guides the Image-Network and the Text-Network to capture multi-label semantic relevance across different modalities. Based on the obtained semantic dictionary, an asymmetric marginscalable loss is proposed to obtain fine-grained pair-wise similarity information, which could contribute to the production of similarity-preserving and discriminative binary codes. Besides, two feature extractors with transformer encoders are designed to achieve the Image-Network and Text-Network, which could extract the representative semantic characteristics from raw heterogeneous samples. Extensive experimental results on various benchmark datasets show that our proposed DSH-FSM framework achieves state-of-the-art crossmodal similarity search performance. Compared to the state-of-the-art methods, the results of mAP are significantly improved by 1.9%, 9.1%, and 9.8%, respectively, on the three widely used datasets.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Asymmetric Margin-scalable Lossmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation