Deep Visual-Semantic Quantization for Efficient Image Retrieval

Cao, Yue; Long, Mingsheng; Wang, Jianmin; Liu, Shichen

doi:10.1109/cvpr.2017.104

Cited by 155 publications

(171 citation statements)

References 22 publications

Supporting

Mentioning

171

Contrasting

Order By: Relevance

“…Figure 5 shows the t-SNE visualizations [31] of the deep representations learned by DVSQ [1], DTQ-2, and DTQ on CIFAR-10 dataset. The deep representations of the proposed DTQ exhibit clear discriminative structures with data points in different categories well separated, while the deep representations by DVSQ [1] exhibit relative vague structures. This validates that by introducing the triplet training to deep quantization, the deep representations generated by our DTQ are more discriminative than that generated by DVSQ, enabling more accurate image retrieval.…”

Section: Tripletmentioning

confidence: 99%

Deep Priority Hashing

Cao

Sun

Long

et al. 2018

Proceedings of the 26th ACM International Conference on Multimedia

Self Cite

View full text Add to dashboard Cite

Deep hashing establishes efficient and effective image retrieval by end-to-end learning of deep representations and hash codes from similarity data. We present a compact coding solution, focusing on deep learning to quantization approach that has shown superior performance over hashing solutions for similarity retrieval. We propose Deep Triplet Quantization (DTQ), a novel approach to learning deep quantization models from the similarity triplets. To enable more effective triplet training, we design a new triplet selection approach, Group Hard, that randomly selects hard triplets in each image group. To generate compact binary codes, we further apply a triplet quantization with weak orthogonality during triplet training. The quantization loss reduces the codebook redundancy and enhances the quantizability of deep representations through back-propagation. Extensive experiments demonstrate that DTQ can generate high-quality and compact binary codes, which yields state-of-the-art image retrieval performance on three benchmark datasets, NUS-WIDE, CIFAR-10, and MS-COCO.

show abstract

Section: Tripletmentioning

confidence: 99%

Deep Priority Hashing

Cao

Sun

Long

et al. 2018

Proceedings of the 26th ACM International Conference on Multimedia

Self Cite

View full text Add to dashboard Cite

show abstract

“…Supervised methods, on the contrary, can well explore the semantic labels for enhancing the cross-modal correlations and reducing the semantic gap [19], hence they generally outperform unsupervised methods for cross-modal search. The latest cross-modal retrieval models via deep learning [21,28,6,24,4] have shown that deep models can distill complex cross-modal correlations more effectively. Despite of the success of deep models for cross-modal search, existing cross-modal retrieval methods are mainly unsupervised and not tailored to hash function learning.…”

Section: Introductionmentioning

confidence: 99%

“…More specifically: (1) We explore the feature correlation by reconstructing the feature vectors of one modality from the corresponding hash codes of another modality, and capture the cross-modal correlations revealed by the feature vectors; (2) We explore the semantic correlation by maximizing the inter-category separation margin and minimizing the intra-category variance, which will produce more discriminative and semantically consistent hash codes; (3) Since cross-modal data (e.g. image and text) are heterogeneous and are difficult to correlate by linear or shallow models [28,6,4], we enhance both cross-modal correlations in a deep architecture, which will make the embedded hash codes generalize better across different modalities. Comprehensive results on large-scale benchmarks show that CAH significantly outperforms state-of-the-art cross-modal hashing methods.…”

Section: Introductionmentioning

confidence: 99%

Correlation Autoencoder Hashing for Supervised Cross-Modal Search

Cao

Long

Wang

et al. 2016

Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval

Self Cite

View full text Add to dashboard Cite

Due to its storage and query efficiency, hashing has been widely applied to approximate nearest neighbor search from large-scale datasets. While there is increasing interest in cross-modal hashing which facilitates cross-media retrieval by embedding data from different modalities into a common Hamming space, how to distill the cross-modal correlation structure effectively remains a challenging problem. In this paper, we propose a novel supervised cross-modal hashing method, Correlation Autoencoder Hashing (CAH), to learn discriminative and compact binary codes based on deep autoencoders. Specifically, CAH jointly maximizes the feature correlation revealed by bimodal data and the semantic correlation conveyed in similarity labels, while embeds them into hash codes by nonlinear deep autoencoders. Extensive experiments clearly show the superior effectiveness and efficiency of CAH against the state-of-the-art hashing methods on standard cross-modal retrieval benchmarks.

show abstract

“…Compared with unsupervised methods [4,19], supervised methods [5,14,17,18,20] can yield better performance with the support of label supervision. With the rapid development of deep neural network, deep hashing methods [1,2,7,11,12,16,21] have demonstrated superior performance over non-deep hashing methods and achieved state-of-the-art results on public benchmarks. However, among mainstream deep hashing frameworks, humanannotated labels purely supervise the distribution alignment of hash code embedding, yet fail to trigger context-aware visual representation learning, let alone optimal binary codes generation.…”

Section: Introductionmentioning

confidence: 99%

Deep Collaborative Discrete Hashing with Semantic-Invariant Structure

Wang

Zhang

Luo

et al. 2019

Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

Existing deep hashing approaches fail to fully explore semantic correlations and neglect the effect of linguistic context on visual attention learning, leading to inferior performance. This paper proposes a dual-stream learning framework, dubbed Deep Collaborative Discrete Hashing (DCDH), which constructs a discriminative common discrete space by collaboratively incorporating the shared and individual semantics deduced from visual features and semantic labels. Specifically, the context-aware representations are generated by employing the outer product of visual embeddings and semantic encodings. Moreover, we reconstruct the labels and introduce the focal loss to take advantage of frequent and rare concepts. The common binary code space is built on the joint learning of the visual representations attended by language, the semantic-invariant structure construction and the label distribution correction. Extensive experiments demonstrate the superiority of our method.

show abstract

Deep Visual-Semantic Quantization for Efficient Image Retrieval

Cited by 155 publications

References 22 publications

Deep Priority Hashing

Deep Priority Hashing

Correlation Autoencoder Hashing for Supervised Cross-Modal Search

Deep Collaborative Discrete Hashing with Semantic-Invariant Structure

Contact Info

Product

Resources

About