Deep Semantic-Preserving Reconstruction Hashing for Unsupervised Cross-Modal Retrieval

Abstract

Deep hashing is the mainstream algorithm for large-scale cross-modal retrieval due to its high retrieval speed and low storage capacity, but the problem of reconstruction of modal semantic information is still very challenging. In order to further solve the problem of unsupervised cross-modal retrieval semantic reconstruction, we propose a novel deep semantic-preserving reconstruction hashing (DSPRH). The algorithm combines spatial and channel semantic information, and mines modal semantic information based on adaptive self-encoding and joint semantic reconstruction loss. The main contributions are as follows: (1) We introduce a new spatial pooling network module based on tensor regular-polymorphic decomposition theory to generate rank-1 tensor to capture high-order context semantics, which can assist the backbone network to capture important contextual modal semantic information. (2) Based on optimization perspective, we use global covariance pooling to capture channel semantic information and accelerate network convergence. In feature reconstruction layer, we use two bottlenecks auto-encoding to achieve visual-text modal interaction. (3) In metric learning, we design a new loss function to optimize model parameters, which can preserve the correlation between image modalities and text modalities. The DSPRH algorithm is tested on MIRFlickr-25K and NUS-WIDE. The experimental results show that DSPRH has achieved better performance on retrieval tasks.

Deep parameter-free attention hashing for image retrieval

“…However, TBIR and CBIR need a great quantity manual operation and computational resources. On the contrary, deep hashing methods 12 – 14 have obvious advantages by utilizing CNN as a features extractor. Existing deep hashing are divided into data-independent and data-dependent.…”

Deep Cross-Dimensional Attention Hashing for Image Retrieval

“…As an important branch of image retrieval algorithms, deep hashing methods are popular because they not only ensure lower storage requirements but also guarantee higher retrieval efficiency [11][12][13][14][15]. Deep hashing methods use deep a convolutional neural network (CNN) to learn the high-dimensional semantic information from an image and convert high-dimensional semantic information into low-dimensional binary code using hash functions [16][17][18][19][20]. Such algorithms can effectively retain the similarity of images, compress the storage cost greatly, and have high-speed retrieval efficiency.…”

“…Such algorithms can effectively retain the similarity of images, compress the storage cost greatly, and have high-speed retrieval efficiency. Data-independent hashing and data-dependent hashing are two types of hashing algorithms, and they are introduced separately in the following article [20][21][22][23].…”

“…For the deep hashing methods, they usually use a CNN to learn the higher-dimensional semantic features of images; then, the hash function is used to map these feature maps to the hash code [18][19][20][21][22][23][24][25]. However, a CNN has the problem that the feature extraction is insufficient and imbalanced.…”