Cross-Media Semantic Matching based on Sparse Representation

Abstract

With the rapid growth of multi-modal data, cross-media retrieval has aroused many research interests. In this paper, the cross-media retrieval includes two tasks: query image retrieves relevant text and query text retrieves relevant images. With the development of sparse representation, two independent sparse representation classifiers are used to map the heterogeneous features of images and texts into their common semantic space before implementing similarity comparison. The proposed method makes full use of semantic information, and it is effective in the retrieving task. The performance of this method was evaluated on Wiki dataset, NUS-WIDE dataset, Wiki dataset with CNN features and Pascal dataset with CNN features. The experimental results validate its effectiveness compared with several state-of-the-art algorithms on the Mean Average Precision and other performance indexes.

Convolutional Neural Network-Based Cross-Media Semantic Matching and User Adaptive Satisfaction Analysis Model

“…Cross-modality means that cross-media data express the same concept, the same semantics, or the same event through different modalities such as image, video, and text. Cross-source refers to cross-media data from other sources but expresses similar semantics [ 4 ]. Cross-space refers to the data that coexist in information space and the physical world with the help of interaction behavior mechanisms.…”