Object-Location-Aware Hashing for Multi-Label Image Retrieval via Automatic Mask Learning

Huang, Changqin; Yang, Shangming; Pan, Yan; Lai, Hanjiang

doi:10.1109/tip.2018.2839522

Cited by 37 publications

(18 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, feature aggregation such as sum/average, max pooling, and embedding (e.g., BoW [51], VLAD [22], FV [35]) is also applied to improve the discriminativeness and compactness of the representations. In fact, representation learning is still an ongoing topic, and numerous methods are proposed in recent years which are conducted either supervised [2,5,14] or non-supervised [41,56], and directly concatenated [4,42] or hash embedded [17,30]. We will skip these methods here because we have skipped representation learning in our framework.…”

Section: Deep Feature Extractionmentioning

confidence: 99%

Indicative Image Retrieval: Turning Blackbox Learning into Grey

Zhang¹,

Yang²,

Tian³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

Section: Deep Feature Extractionmentioning

confidence: 99%

Indicative Image Retrieval: Turning Blackbox Learning into Grey

Zhang¹,

Yang²,

Tian³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Recent techniques mainly focus on two parts: deep network architectures and training algorithms. The deep network architectures include single feedforward pass models [26], multiple feedforward pass models [27], attention based models [28], and deep hashing embedding based models [29]. While the training algorithms focus on classification based learning [30], metric based learning [31], and unsupervised-based learning [32].…”

Section: A Image Retrievalmentioning

confidence: 99%

Task-Oriented Multi-User Semantic Communications

Xie¹,

Qin²,

Tao³

et al. 2021

Preprint

View full text Add to dashboard Cite

While semantic communications have shown the potential in the case of single-modal single-users, its applications to the multi-user scenario remain limited. In this paper, we investigate deep learning (DL) based multi-user semantic communication systems for transmitting single-modal data and multimodal data, respectively. We will adopt three intelligent tasks, including, image retrieval, machine translation, and visual question answering (VQA) as the transmission goal of semantic communication systems. We will then propose a Transformer based unique framework to unify the structure of transmitters for different tasks. For the single-modal multi-user system, we will propose two Transformer based models, named, DeepSC-IR and DeepSC-MT, to perform image retrieval and machine translation, respectively. In this case, DeepSC-IR is trained to optimize the distance in embedding space between images and DeepSC-MT is trained to minimize the semantic errors by recovering the semantic meaning of sentences. For the multimodal multi-user system, we develop a Transformer enabled model, named, DeepSC-VQA, for the VQA task by extracting text-image information at the transmitters and fusing it at the receiver. In particular, a novel layer-wise Transformer is designed to help fuse multimodal data by adding connection between each of the encoder and decoder layers. Numerical results will show that the proposed models are superior to traditional communications in terms of the robustness to channels, computational complexity, transmission delay, and the task-execution performance at various task-specific metrics.

show abstract

“…Lai et al [33] learned instance-aware representations for the image data organized in groups for multiple labels with the assistant of Spatial Pyramid Pooling (SPP) layer [67]. Huang et al [68] proposed to focus on the objects instead of the background in an image by learning a binary mask map to identify the location of the objects.…”

Section: Related Workmentioning

confidence: 99%

Rank-Consistency Deep Hashing for Scalable Multi-Label Image Search

Zhou

2021

IEEE Trans. Multimedia

View full text Add to dashboard Cite

As hashing becomes an increasingly appealing technique for large-scale image retrieval, multi-label hashing is also attracting more attention for the ability to exploit multi-level semantic contents. In this paper, we propose a novel deep hashing method for scalable multi-label image search. Unlike existing approaches with conventional objectives such as contrast and triplet losses, we employ a rank list, rather than pairs or triplets, to provide sufficient global supervision information for all the samples. Specifically, a new rank-consistency objective is applied to align the similarity orders from two spaces, the original space and the hamming space. A powerful loss function is designed to penalize the samples whose semantic similarity and hamming distance are mismatched in two spaces. Besides, a multi-label softmax cross-entropy loss is presented to enhance the discriminative power with a concise formulation of the derivative function. In order to manipulate the neighborhood structure of the samples with different labels, we design a multi-label clustering loss to cluster the hashing vectors of the samples with the same labels by reducing the distances between the samples and their multiple corresponding class centers. The stateof-the-art experimental results achieved on three public multilabel datasets, MIRFLICKR-25K, IAPRTC12 and NUS-WIDE, demonstrate the effectiveness of the proposed method.

show abstract

Object-Location-Aware Hashing for Multi-Label Image Retrieval via Automatic Mask Learning

Cited by 37 publications

References 21 publications

Indicative Image Retrieval: Turning Blackbox Learning into Grey

Indicative Image Retrieval: Turning Blackbox Learning into Grey

Task-Oriented Multi-User Semantic Communications

Rank-Consistency Deep Hashing for Scalable Multi-Label Image Search

Contact Info

Product

Resources

About