Rank-Consistency Deep Hashing for Scalable Multi-Label Image Search

Abstract

As hashing becomes an increasingly appealing technique for large-scale image retrieval, multi-label hashing is also attracting more attention for the ability to exploit multi-level semantic contents. In this paper, we propose a novel deep hashing method for scalable multi-label image search. Unlike existing approaches with conventional objectives such as contrast and triplet losses, we employ a rank list, rather than pairs or triplets, to provide sufficient global supervision information for all the samples. Specifically, a new rank-consistency objective is applied to align the similarity orders from two spaces, the original space and the hamming space. A powerful loss function is designed to penalize the samples whose semantic similarity and hamming distance are mismatched in two spaces. Besides, a multi-label softmax cross-entropy loss is presented to enhance the discriminative power with a concise formulation of the derivative function. In order to manipulate the neighborhood structure of the samples with different labels, we design a multi-label clustering loss to cluster the hashing vectors of the samples with the same labels by reducing the distances between the samples and their multiple corresponding class centers. The stateof-the-art experimental results achieved on three public multilabel datasets, MIRFLICKR-25K, IAPRTC12 and NUS-WIDE, demonstrate the effectiveness of the proposed method.

HyP$^2$ Loss: Beyond Hypersphere Metric Space for Multi-label Image Retrieval

“…Time Complexity Analysis. The proposed method converges faster and is proven more efficient and stable than those pair-based methods [18,25,32,57,58] (as we will justify in Fig. 3).…”

“…Pair-based Methods. Pair-based methods [18,25,32,51,57,58] are predominant in multi-label retrieval [37], which focus on exploring data-to-data relations from the paired samples through metric learning. In the field of image retrieval, Constrictive loss [7,15] innovatively determines the gradient descent directions by estimating the similarity between feature vector pairs.…”

“…IAH [25] divides hash codes into groups to separately excavate instanceaware image representations. DMSSPH [51] and RCDH [32] further improve the performance by considering the semantic similarity by supervision on grouped labels and additional regularization.…”

“…standard baselines, including HashNet [5], DMSSPH [51], DQN [4], DLBHC [29], DHN [59], NINH [24], DSH [30], and IAH [25], 2). state-of-the-art deep hashing methods, including IDHN [57], Proxy Anchor [21], CSQ [55], OrthoHash [17], OLAH [18], and DCILH [33]. Note that OLAH and DCILH are two state-of-the-art deep hashing methods specifically designed for multi-label image retrieval.…”

Determination of multi-modal dimension metric learning with application to web image retrieval

“…The "Image CLEF" dataset3, which was utilized in [31], is the third testbed. This medical picture collection contains 7,157 photographs divided into 20 categories.…”