Correlation Autoencoder Hashing for Supervised Cross-Modal Search

Cao, Yue; Long, Mingsheng; Wang, Jianmin; Han, Zhu

doi:10.1145/2911996.2912000

Cited by 90 publications

(42 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This may be due to several reasons. First, the method of CAH [17] still uses handcrafted image features as the input to their deep neural networks whereas our model starts learning from raw images. In the methods of DVSH [13] and CMDVH [14], the modality-specific hash functions are learned such that the non-linear relationship of samples from different modalities are exploited.…”

Section: ) Comparison To Deep Cross-modal Hashing Methodsmentioning

confidence: 99%

“…• CMNN [15]: The method is to learn a similarity preserving network for cross-modalities through a coupled Siamese network with hinge loss. • CAH [17]: A model that is designed with a stacked autoencoder architecture to jointly maximize the feature and semantic correlation across modalities. • DCMH [18]: It is an end-to-end deep learning framework with a negative log likelihood criterion to preserve the similarity between real-value representations having the same class.…”

Section: B Competitors and Evaluation Setupmentioning

confidence: 99%

See 1 more Smart Citation

Cycle-Consistent Deep Generative Hashing for Cross-Modal Retrieval

Wang

Shao

2019

IEEE Trans. on Image Process.

246

108

View full text Add to dashboard Cite

In this paper, we propose a novel deep generative approach to cross-modal retrieval to learn hash functions in the absence of paired training samples through the cycle consistency loss. Our proposed approach employs adversarial training scheme to lean a couple of hash functions enabling translation between modalities while assuming the underlying semantic relationship. To induce the hash codes with semantics to the inputoutput pair, cycle consistency loss is further proposed upon the adversarial training to strengthen the correlations between inputs and corresponding outputs. Our approach is generative to learn hash functions such that the learned hash codes can maximally correlate each input-output correspondence, meanwhile can also regenerate the inputs so as to minimize the information loss. The learning to hash embedding is thus performed to jointly optimize the parameters of the hash functions across modalities as well as the associated generative models. Extensive experiments on a variety of large-scale cross-modal data sets demonstrate that our proposed method achieves better retrieval results than the state-of-the-arts.

show abstract

Section: ) Comparison To Deep Cross-modal Hashing Methodsmentioning

confidence: 99%

Section: B Competitors and Evaluation Setupmentioning

confidence: 99%

Cycle-Consistent Deep Generative Hashing for Cross-Modal Retrieval

Wang

Shao

2019

IEEE Trans. on Image Process.

246

108

View full text Add to dashboard Cite

show abstract

“…Supervised hashing methods [2,4,14,16,39,40,44] aim to exploit available supervised information (such as labels or the semantic affinities of training data) to improve performance. Brostein et al [2] present a cross-modal hashing approach by preserving the intra-class similarity via eigendecomposition and boosting.…”

Section: Related Workmentioning

confidence: 99%

“…In fact, in standard cross-modal benchmark datasets such as NUS-WIDE [6] and Microsoft COCO [15], an image instance can be assigned to multiple category labels [27], which is beneficial as it permits semantic relevance to be described more accurately across different modalities. Second, these methods enforce a narrowing of the modality gap by constraining the corresponding hash codes with certain pre-defined loss functions [4]. The code length is usually text A collection of baseball players who are at home plate.…”

Section: Introductionmentioning

confidence: 99%

Self-Supervised Adversarial Hashing Networks for Cross-Modal Retrieval

Deng

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

395

206

View full text Add to dashboard Cite

Thanks to the success of deep learning, cross-modal retrieval has made significant progress recently. However, there still remains a crucial bottleneck: how to bridge the modality gap to further enhance the retrieval accuracy. In this paper, we propose a self-supervised adversarial hashing (SSAH) approach, which lies among the early attempts to incorporate adversarial learning into cross-modal hashing in a self-supervised fashion. The primary contribution of this work is that two adversarial networks are leveraged to maximize the semantic correlation and consistency of the representations between different modalities. In addition, we harness a self-supervised semantic network to discover high-level semantic information in the form of multi-label annotations. Such information guides the feature learning process and preserves the modality relationships in both the common semantic space and the Hamming space. Extensive experiments carried out on three benchmark datasets validate that the proposed SSAH surpasses the state-of-the-art methods.

show abstract

“…Deep cross-modal hashing (DCMH) [24] combines hashing learning and deep feature learning by preserving the semantic similarity between modalities. Correlation autoencoder hashing (CAH) [28] embeds the maximum cross-modal similarity into hash codes using nonlinear deep autoencoders. Correlation hashing network (CHN) [29] jointly learns image and text representations tailored to hash coding and formally controls the quantization error.…”

Section: Related Workmentioning

confidence: 99%

Cross-Modal Zero-Shot Hashing

Liu

Zhao

Wang

et al. 2019

2019 IEEE International Conference on Data Mining (ICDM)

View full text Add to dashboard Cite

Hashing has been widely studied for big data retrieval due to its low storage cost and fast query speed. Zero-shot hashing (ZSH) aims to learn a hashing model that is trained using only samples from seen categories, but can generalize well to samples of unseen categories. ZSH generally uses category attributes to seek a semantic embedding space to transfer knowledge from seen categories to unseen ones. As a result, it may perform poorly when labeled data are insufficient. ZSH methods are mainly designed for single-modality data, which prevents their application to the widely spread multi-modal data. On the other hand, existing cross-modal hashing solutions assume that all the modalities share the same category labels, while in practice the labels of different data modalities may be different. To address these issues, we propose a general Cross-modal Zero-shot Hashing (CZHash) solution to effectively leverage unlabeled and labeled multi-modality data with different label spaces. CZHash first quantifies the composite similarity between instances using label and feature information. It then defines an objective function to achieve deep feature learning compatible with the composite similarity preserving, category attribute space learning, and hashing coding function learning. CZHash further introduces an alternative optimization procedure to jointly optimize these learning objectives. Experiments on benchmark multi-modal datasets show that CZHash significantly outperforms related representative hashing approaches both on effectiveness and adaptability.

show abstract

Correlation Autoencoder Hashing for Supervised Cross-Modal Search

Cited by 90 publications

References 28 publications

Cycle-Consistent Deep Generative Hashing for Cross-Modal Retrieval

Cycle-Consistent Deep Generative Hashing for Cross-Modal Retrieval

Self-Supervised Adversarial Hashing Networks for Cross-Modal Retrieval

Cross-Modal Zero-Shot Hashing

Contact Info

Product

Resources

About