Label-Attended Hashing for Multi-Label Image Retrieval

Abstract

<jats:p>For the multi-label image retrieval, the existing hashing algorithms neglect the dependency between objects and thus fail to capture the attention information in the feature extraction, which affects the precision of hash codes. To address this problem, we explore the inter-dependency between objects through their co-occurrence correlation from the label set and adopt Multi-modal Factorized Bilinear (MFB) pooling component so that the image representation learning can capture this attention information. We propose a Label-Attended Hashing (LAH) algorithm which enables an end-to-end hash model with inter-dependency feature extraction. LAH first combines Convolutional Neural Network (CNN) and Graph Convolution Network (GCN) to separately generate the image representation and label co-occurrence embeddings, then adopts MFB to fuse these two modal vectors, finally learns the hash function with a Cauchy distribution based loss function via back propagation. Extensive experiments on public multi-label datasets demonstrate that (1) LAH can achieve the state-of-the-art retrieval results and (2) the usage of co-occurrence relationship and MFB not only promotes the precision of hash codes but also accelerates the hash learning. GitHub address: https://github.com/IDSM-AI/LAH.</jats:p>

A method for semantic-based image retrieval using hierarchical clustering tree and graph

“…This proves that our proposed method is effective. Cao, 2018 [25] 0.857645 Y. Xie, 2020 [26] 0.862848 SIR-SgGP 0.875467…”

Revisiting Performance Measures for Cross-Modal Hashing

“…The co-occurrence probability [21] represents the probability that every two labels appear together in each item. We define the matrix S as the co-occurrence probability matrix.…”

Gat-Assisted Deep Hashing For Multi-Label Image Retrieval

“…We compare the retrieval performance of DHGAT with existing deep hashing methods and multi-label methods, deep hashing methods include DCH [13], GCNH [14], MMHH [11], multi-label methods include RCDH [5], LAH [6].…”

“…Other multi-label hashing methods, such as LAH [6] and NRDH [7] use the Graph Convolutional network (GCN) to extract the label feature, and then merge the image-level and label-level features to greatly improve the model's ability to generate distinguishable hash code, so as to improve the performance of the model. These methods apply GCN, which relies on the overall dataset's distribution, to extract the label features, the ignorance of label co-occurrence relationship further leads to incomplete exploitation of different labels' weights, and affect the label features' expression finally.…”