Ranking-Based Deep Cross-Modal Hashing

Liu, Xuanwu; Yu, Guoxian; Domeniconi, Carlotta; Wang, Jun; Ren, Yazhou; Guo, Maozu

doi:10.1609/aaai.v33i01.33014400

Cited by 63 publications

(22 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Like single-modal hashing methods which are based on a structure preserving criterion, existing cross-modal hashing can be categorized into three types: pairwise [46], [20], [33], [28], [19], [32], [9], [23], [8], [14], [4], multi-wise [37], [42], [26], and implicit similarity preserving [15], [34]. Semantic correlation maximization (SCM) [46] optimizes the hashing functions by maximizing the correlation between two modalities with respect to the pairwise semantic similarity.…”

Section: Related Workmentioning

confidence: 99%

“…Deep Cross-Modal Hashing (DCMH) [13] combines hashing learning and deep feature learning by preserving the semantic similarity between modalities. Rankingbased Deep Cross-modal Hashing (RDCMH) [26] preserves the multi-level semantic similarity order between labeled and unlabeled multi-label samples for cross-modal hashing. Deep semantic-preserving ordinal hashing (DSPOH) [14] adopts deep neural networks to learn hashing functions by exploring the ranking structure.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Flexible Cross-Modal Hashing

Liu

Wang

et al. 2022

IEEE Trans. Neural Netw. Learning Syst.

Self Cite

View full text Add to dashboard Cite

Hashing has been widely adopted for large-scale data retrieval in many domains, due to its low storage cost and high retrieval speed. Existing cross-modal hashing methods optimistically assume that the correspondence between training samples across modalities is readily available. This assumption is unrealistic in practical applications. In addition, existing methods generally require the same number of samples across different modalities, which restricts their flexibility.We propose a flexible cross-modal hashing approach (Flex-CMH) to learn effective hashing codes from weakly-paired data, whose correspondence across modalities is partially (or even totally) unknown. FlexCMH first introduces a clustering-based matching strategy to explore the structure of each cluster, and thus to find the potential correspondence between clusters (and samples therein) across modalities. To reduce the impact of an incomplete correspondence, it jointly optimizes the potential correspondence, the cross-modal hashing functions derived from the correspondence, and a hashing quantitative loss in a unified objective function. An alternative optimization technique is also proposed to coordinate the correspondence and hash functions, and to reinforce the reciprocal effects of the two objectives. Experiments on public multi-modal datasets show that FlexCMH achieves significantly better results than state-of-the-art methods, and it indeed offers a high degree of flexibility for practical crossmodal hashing tasks.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Flexible Cross-Modal Hashing

Liu

Wang

et al. 2022

IEEE Trans. Neural Netw. Learning Syst.

Self Cite

View full text Add to dashboard Cite

show abstract

“…We collected the multiple features of these images from [53], where each image is represented by six representative feature views: HUE, SIFT, GIST, HSV, RGB and LAB. Each sample in Mirflicker 2 and Nus-wide 3 consists of an image and textual tags, we construct the two views (image and text) according to [54]. For each dataset, we randomly sample 30% data for training and use the remaining 70% data for testing (unlabeled data).…”

Section: A Experimental Setupmentioning

confidence: 99%

Individuality- and Commonality-Based Multiview Multilabel Learning

Tan

Wang

et al. 2021

IEEE Trans. Cybern.

Self Cite

View full text Add to dashboard Cite

In multi-view multi-label learning, each object is represented by several heterogeneous feature representations and is also annotated with a set of discrete non-exclusive labels. Previous studies typically focus on capturing the shared latent patterns among multiple views, while do not sufficiently consider the diverse characteristics of individual views, which can cause performance degradation. In this paper, we propose a novel approach (ICM2L) to explicitly explore the individuality and commonality information of multi-label multiple view data in a unified model. Specifically, a common subspace is learned across different views to capture the shared patterns. Then, multiple individual classifiers are exploited to explore the characteristics of individual views. Next, an ensemble strategy is adopted to make prediction. Finally, we develop an alternative solution to jointly optimize our model, which can enhance the robustness of the proposed model towards rare labels and reinforce the reciprocal effects of individuality and commonality among heterogeneous views, and thus further improve the performance. Experiments on various real-word datasets validate the effectiveness of ICM2L against state-of-the-art solutions, and ICM2L can leverage the individuality and commonality information to achieve an improved performance as well as to enhance the robustness toward rare labels.

show abstract

“…Song et al [17] combined global context with locally-guided features with multi-head selfattention and residual learning to improve cross-modal representation. Liu et al [18] proposed ranking-based deep crossmodal hashing method that integrates the semantic ranking information.…”

Section: Related Workmentioning

confidence: 99%

Multi-Graph Based Hierarchical Semantic Fusion for Cross-Modal Representation

Zhu

Zhang

Song

et al. 2021

2021 IEEE International Conference on Multimedia and Expo (ICME)

View full text Add to dashboard Cite

The main challenge of cross-modal retrieval is how to efficiently realize semantic alignment and reduce the heterogeneity gap. However, existing approaches ignore the multigrained semantic knowledge learning from different modalities. To this end, this paper proposes a novel end-to-end cross-modal representation method, termed as Multi-Graph based Hierarchical Semantic Fusion (MG-HSF). This method is an integration of multi-graph hierarchical semantic fusion with cross-modal adversarial learning, which captures finegrained and coarse-grained semantic knowledge from crossmodal samples, and generate modalities-invariant representations in a common subspace. To evaluate the performance, extensive experiments are conducted on three benchmarks. The experimental results show that our method is superior than the state-of-the-arts.

show abstract

Ranking-Based Deep Cross-Modal Hashing

Cited by 63 publications

References 13 publications

Flexible Cross-Modal Hashing

Flexible Cross-Modal Hashing

Individuality- and Commonality-Based Multiview Multilabel Learning

Multi-Graph Based Hierarchical Semantic Fusion for Cross-Modal Representation

Contact Info

Product

Resources

About