Unsupervised Deep Cross-Modal Hashing by Knowledge Distillation for Large-scale Cross-modal Retrieval

Li, Mingyong; Wang, Hongya

doi:10.1145/3460426.3463626

Cited by 18 publications

(2 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…DGCPN [10] explores intrinsic semantic relationships with graph-neighbor coherence to avoid suboptimal retrieval Hamming space. With introducing knowledge distillation scheme, KDCMH [11] trains an unsupervised method as the teacher model used to provide distillation information to guide supervised method. CMIMH [12] tries to find a balance between reducing the modality gap and losing modality-private information by maximizing mutual information.…”

Section: A Non-continuous Cross-modal Hashingmentioning

confidence: 99%

Multi-Manifold Deep Discriminative Cross-Modal Hashing for Medical Image Retrieval

Zeng

Zheng

et al. 2022

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

Hashing methods have made significant progress in cross-modal retrieval tasks with fast query speed and low storage cost. Among them, deep learning-based hashing achieves better performance on large-scale data due to its excellent extraction and representation ability for nonlinear heterogeneous features. However, there are still two main challenges in catastrophic forgetting when data with new categories arrive continuously, and time-consuming for non-continuous hashing retrieval to retrain for updating. To this end, we, in this paper, propose a novel deep lifelong cross-modal hashing to achieve lifelong hashing retrieval instead of re-training hash function repeatedly when new data arrive. Specifically, we design lifelong learning strategy to update hash functions by directly training the incremental data instead of retraining new hash functions using all the accumulated data, which significantly reduce training time. Then, we propose lifelong hashing loss to enable original hash codes participate in lifelong learning but remain invariant, and further preserve the similarity and dis-similarity among original and incremental hash codes to maintain performance. Additionally, considering distribution heterogeneity when new data arriving continuously, we introduce multi-label semantic similarity to supervise hash learning, and it has been proven that the similarity improves performance with detailed analysis. Experimental results on benchmark datasets show that the proposed methods achieves comparative performance comparing with recent state-of-the-art cross-modal hashing methods, and it yields substantial average increments over 20% in retrieval accuracy and almost reduces over 80% training time when new data arrives continuously.

show abstract

Section: A Non-continuous Cross-modal Hashingmentioning

confidence: 99%

Multi-Manifold Deep Discriminative Cross-Modal Hashing for Medical Image Retrieval

Zeng

Zheng

et al. 2022

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

show abstract

“…4, it can be observed that on both I T and T I, our proposed KGUCH achieves the best performance among the other compared methods in precision-recall curves. CVH (IJCAI 2011) [8] 0.458 0.432 0.410 0.474 0.445 0.419 0.466 0.439 0.415 PDH (ICML 2013) [9] 0.475 0.484 0.480 0.489 0.512 0.507 0.482 0.498 0.494 CMFH (TIP 2016) [25] 0.517 0.550 0.547 0.439 0.416 0.377 0.478 0.483 0.462 CCQ (SIGIR 2016) [26] 0.504 0.505 0.506 0.499 0.496 0.492 0.502 0.501 0.499 CMSSH (CVPR 2010) [15] 0.512 0.470 0.479 0.519 0.498 0.456 0.516 0.484 0.468 SCM (AAAI 2014) [16] 0.517 0.514 0.518 0.518 0.510 0.517 0.518 0.512 0.518 DJSRH (CVPR 2019) [13] 0.513 0.535 0.566 0.546 0.568 0.599 0.530 0.552 0.583 JDSH (SIGIR 2020) [27] 0.554 0.561 0.582 0.582 0.596 0.626 0.568 0.579 0.604 KDCMH (ICMR 2021) [28] 0.615 0.628 0.637 0.623 0.636 0.647 0.619 0.632 0.642 DGCPN (AAAI 2021) [14] 0 CVH (IJCAI 2011) [8] 0.601 0.585 0.576 0.605 0.590 0.583 0.603 0.588 0.580 PDH (ICML 2013) [9] 0.622 0.623 0.619 0.625 0.626 0.629 0.624 0.625 0.624 CMFH (TIP 2016) [25] 0.661 0.658 0.661 0.609 0.604 0.573 0.635 0.631 0.617 CCQ (SIGIR 2016) [26] 0.636 0.638 0.637 0.625 0.626 0.620 0.631 0.632 0.629 CMSSH (CVPR 2010) [15] 0.609 0.604 0.561 0.611 0.605 0.591 0.610 0.605 0.576 SCM (AAAI 2014) [16] 0.636 0.640 0.641 0.661 0.660 0.668 0.649 0.650 0.655 DJSRH (CVPR 2019) [13] 0.666 0.678 0.699 0.683 0.694 0.717 0.675 0.686 0.708 JDSH (SIGIR 2020) [27] 0.669 0.683 0.698 0.686 0.699 0.716 0.678 0.691 0.707 KDCMH (ICMR 2021) [28] 0.713 0.716 0.724 0.711 0.715 0.731 0.712 0.716 0.728 DGCPN (AAAI 2021) [14] 0 [25] 0.163 0.176 0.163 0.495 0.513 0.533 0.329 0.344 0.348 CCQ (SIGIR 2016) [26] 0.235 0.237 0.237 0.400 0.441 0.457 0.317 0.339 0.347 SCM (AAAI 2014) [16] 0.237 0.238 0.238 0.370 0.424 0.437 0.304 0.331 0.338 DJSRH (CVPR 2019) [13] 0.384 0.398 0.406 0.512 0.536 0.544 0.448 0.467 0.475 JDSH (SIGIR 2020) [27] 0.351 0.383 0.399 0.398 0.448 0.480 0.375 0.416 0.440 DGCPN (AAAI 2021) [14] 0 The results are shown in Table 4. Compared to the proposed approach KGUCH with KGUCH-Baseline, constructing a semantic graph in our approach can significantly help promot...…”

mentioning

confidence: 99%

Unsupervised Cross‐Media Hashing Learning via Knowledge Graph

Peng

2022

Chinese J of Electronics

View full text Add to dashboard Cite

With the rapid growth of multimedia data, cross‐media hashing has become an important technology for fast cross‐media retrieval. Because the manual annotations are difficult to obtain in real‐world application, unsupervised cross‐media hashing is studied to address the hashing learning without manual annotations. Existing unsupervised cross‐media hashing methods generally focus on calculating the similarities through the features of multimedia data, while the learned hashing code cannot reflect the semantic relationship among the multimedia data, which hinders the accuracy in the cross‐media retrieval. When humans try to understand multimedia data, the knowledge of concept relations in our brain plays an important role in obtaining high‐level semantic. Inspired by this, we propose a knowledge guided unsupervised cross‐media hashing (KGUCH) approach, which applies the knowledge graph to construct high‐level semantic correlations for unsupervised cross‐media hash learning. Our contributions in this paper can be summarized as follows: 1) The knowledge graph is introduced as auxiliary knowledge to construct the semantic graph for the concepts in each image and text instance, which can bridge the multimedia data with high‐level semantic correlations to improve the accuracy of learned hash codes for cross‐media retrieval. 2) The proposed KGUCH approach constructs correlation of the multimedia data from both the semantic and the feature aspects, which can exploit complementary information to promote the unsupervised cross‐media hash learning. The experiments are conducted on three widely‐used datasets, which verify the effectiveness of our proposed KGUCH approach.

show abstract

Heterogeneous Interactive Learning Network for Unsupervised Cross-Modal Retrieval

Zheng¹,

Zhang²

2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Unsupervised Deep Cross-Modal Hashing by Knowledge Distillation for Large-scale Cross-modal Retrieval

Cited by 18 publications

References 29 publications

Multi-Manifold Deep Discriminative Cross-Modal Hashing for Medical Image Retrieval

Multi-Manifold Deep Discriminative Cross-Modal Hashing for Medical Image Retrieval

Unsupervised Cross‐Media Hashing Learning via Knowledge Graph

Heterogeneous Interactive Learning Network for Unsupervised Cross-Modal Retrieval

Contact Info

Product

Resources

About