Image Retrieval with Structured Object Queries Using Latent Ranking SVM

Abstract

Abstract. We consider image retrieval with structured object queriesqueries that specify the objects that should be present in the scene, and their spatial relations. An example of such queries is "car on the road". Existing image retrieval systems typically consider queries consisting of object classes (i.e. keywords). They train a separate classifier for each object class and combine the output heuristically. In contrast, we develop a learning framework to jointly consider object classes and their relations. Our method considers not only the objects in the query ("car" and "road" in the above example), but also related object categories can be useful for retrieval. Since we do not have ground-truth labeling of object bounding boxes on the test image, we represent them as latent variables in our model. Our learning method is an extension of the ranking SVM with latent variables, which we call latent ranking SVM. We demonstrate image retrieval and ranking results on a dataset with more than a hundred of object classes.

Towards brain-activity-controlled information retrieval: Decoding image relevance from MEG signals

“…Hence, the computational tools for implicit relevance prediction are classifiers that take as inputs a feature representation for each item and then produce as the output the probabilities of the two classes. Alternatively, relevance feedback could also be provided by ranking the items according to the estimated relevance, for example by simply sorting them according to the probabilities of relevance, or by directly estimating the ranks (Huang et al, 2010;Lan et al, 2012).…”

Image with a Message: Towards Detecting Non-Literal Image Usages by Visual Linking

“…Then the score H θ (Q, S) between a query Q and a segment S, can be obtained by maximizing over M (Lan et al, 2012;Joachims, 2002):…”

Multi-Modal Image Retrieval for Complex Queries using Small Codes

“…To this end, there have been major advances in the field of image retrieval in the last few years. For example, image retrieval has progressed from retrieving images based on single label queries [1], [7] to multi-label queries [9] [10], [16], [30] and structured queries [19].…”