Cross-Media Similarity Evaluation for Web Image Retrieval in the Wild

Abstract

In order to retrieve unlabeled images by textual queries, cross-media similarity computation is a key ingredient. Although novel methods are continuously introduced, little has been done to evaluate these methods together with large-scale query log analysis. Consequently, how far have these methods brought us in answering real-user queries is unclear. Given baseline methods that use relatively simple text/image matching, how much progress have advanced models made is also unclear. This paper takes a pragmatic approach to answering the two questions. Queries are automatically categorized according to the proposed query visualness measure, and later connected to the evaluation of multiple cross-media similarity models on three test sets. Such a connection reveals that the success of the stateof-the-art is mainly attributed to their good performance on visual-oriented queries, which account for only a small part of real-user queries. To quantify the current progress, we propose a simple text2image method, representing a novel query by a set of images selected from large-scale query log. Consequently, computing cross-media similarity between the query and a given image boils down to comparing the visual similarity between the given image and the selected images. Image retrieval experiments on the challenging Clickture dataset show that the proposed text2image is a strong baseline, comparing favorably to recent deep learning alternatives.

Fine-Grained Fashion Similarity Learning by Attribute-Specific Embedding Network

“…Metrics For the task of attribute-specific fashion retrieval, we report the Mean Average Precision (MAP), a popular performance metric in many retrieval-related tasks (Awad et al 2018;Dong, Li, and Xu 2018). For the triplet relation prediction task, we utilize the prediction accuracy as the metric.…”

Multi-Order Adversarial Representation Learning for Composed Query Image Retrieval

“…Image retrieval is an important research topic in the field of computer vision and multimedia, and has a wide range of applications [1,2], such as fashion retrieval [3,4] and person re-identification [5,6]. The retrieval system usually only uses a picture or a paragraph of text as input.…”