Contrastive Label Correlation Enhanced Unified Hashing Encoder for Cross-modal Retrieval

Multi-Label Weighted Contrastive Cross-Modal Hashing

“…[62] analyzes the relationship between model performance and temperature τ in contrastive learning, and models with small or large temperatures achieve sub-optimal performance. Inspired by the great success of contrastive learning, some contrastive hashing methods [30,54] have been proposed to learn binary representations from multi-modal data and achieved promising performance. UCCH [54] is the first attempt to use contrastive learning in unsupervised cross-modal hashing, which proposes a cross-modal ranking learning loss (CRL) to mitigate the impact of false-negative pairs.…”

“…Unfortunately, due to the explosive growth of data, the computational complexity of real-valued cross-modal retrieval has become an unavoidable challenge. A viable solution is cross-modal hashing [20][21][22][23][24][25][26][27][28][29][30], which maps high-dimensional multi-modal features into compact binary codes and the cross-modal similarity can be calculated by XOR operation efficiently.…”

“…Unlike traditional single-label hashing methods that focus on binary similarity for individual instances, they use multiple labels to construct a semantic similarity matrix so as to learn more accurate similaritiy relationships (e.g., each similarity is defined as a real value between −1 and 1). Meanwhile, motivated by the remarkable achievements in contrastive learning, some contrastive learning methods [30,54] for cross-modal hashing are introduced which aim to capture cross-modal similarities more effectively by comparing samples across modalities.…”

“…Motivation. Recently, some great progress has been made in cross-modal hashing with contrastive learning, such as [30,54]. UCCH [54] represents the initial endeavor in employing contrastive learning within unsupervised cross-modal hashing.…”

“…It introduces a cross-modal ranking learning loss (CRL) to alleviate the influence of false-negative pairs. Conversely, Unihash [30] leverages a contrastive label correlation learning (CLC) loss to establish connections between diverse modalities through category labels. However, they simply apply InfoNCE [55,56] in cross-modal hashing, which treats an image-text pair as a positive sample; otherwise, it is a negative sample.…”