Content-Based Image Retrieval over the Web Using Query by Sketch and Relevance Feedback

Sciascio, Eugenio Di; Mingolla, G.; Mongiello, Marina

doi:10.1007/3-540-48762-x_16

Cited by 43 publications

(29 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Prior work, more closely aligned with our proposed approach, has explored the sketching of line-art shape depictions for SBIR [26] using edge maps to approximate a sketch from a photograph prior to matching. Matusiak et al [27] applied curvature scale space (CSS) [28] as a robust contour representation, although this required a pre-process to first extract the contour.…”

Section: Related Workmentioning

confidence: 99%

A performance evaluation of gradient field HOG descriptor for sketch based image retrieval

Collomosse

2013

Computer Vision and Image Understanding

336

285

View full text Add to dashboard Cite

We present an image retrieval system for the interactive search of photo collections using free-hand sketches depicting shape. We describe Gradient Field HOG (GF-HOG); an adapted form of the HOG descriptor suitable for sketch based image retrieval (SBIR). We incorporate GF-HOG into a Bag of Visual Words (BoVW) retrieval framework, and demonstrate how this combination may be harnessed both for robust SBIR, and for localizing sketched objects within an image. We evaluate over a large Flickr sourced dataset comprising 33 shape categories, using queries from 10 non-expert sketchers. We compare GF-HOG against state-of-the-art descriptors with common distance measures and language models for image retrieval, and explore how affine deformation of the sketch impacts search performance. GF-HOG is shown to consistently outperform retrieval versus SIFT, multi-resolution HOG, Self Similarity, Shape Context and Structure Tensor. Further, we incorporate semantic keywords in to our GF-HOG system to enable the use of annotated sketches for image search. A novel graph-based measure of semantic similarity is proposed and two applications explored: semantic sketch based image retrieval and a semantic photo montage.

show abstract

Section: Related Workmentioning

confidence: 99%

A performance evaluation of gradient field HOG descriptor for sketch based image retrieval

Collomosse

2013

Computer Vision and Image Understanding

336

285

View full text Add to dashboard Cite

show abstract

“…Early sketch based image retrieval (SBIR) systems accepted queries comprising blobs of coloured texture, matched through region adjacency and topology [5,6], shape [7], or spectral descriptors such as wavelets [8]. More recently, SBIR has been applied to large scale (>1 million record) retrieval by matching line-art query sketches to edge information within photographs [9,10,11,12].…”

Section: Related Workmentioning

confidence: 99%

Markov random fields for sketch based video retrieval

James

Wang

et al. 2013

Proceedings of the 3rd ACM Conference on International Conference on Multimedia Retrieval

View full text Add to dashboard Cite

We describe a new system for searching video databases using free-hand sketched queries. Our query sketches depict both object appearance and motion, and are annotated with keywords that indicate the semantic category of each object. We parse space-time volumes from video to form graph representation, which we match to sketches under a Markov Random Field (MRF) optimization. The MRF energy function is used to rank videos for relevance and contains unary, pairwise and higher-order potentials that reflect the colour, shape, motion and type of sketched objects. We evaluate performance over a dataset of 500 sports footage clips.

show abstract

“…Common approaches apply a linear SVM to rerank results [26] more recent approaches are applied over multiple features using multiple classifiers [32] or kernels [33]. Within SBR, image based RF techniques have been demonstrated [22,12] yet, despite the well-known advantages of RF when dealing with complex multi-model datasets, RF has not yet been explored for SBVR.…”

Section: Related Workmentioning

confidence: 99%

“…This is essential given the ambiguity inherent in sketch (e. g. shape) when faced with a large database, and given the multiple-modalities (e. g. colour, motion, shape, semantics) present within a hybrid sketched query. Although common in the wider information retrieval literature, and fleetingly explored for SBIR [22,12], relevance feedback has not been seen in any SBVR system before.…”

Section: Introductionmentioning

confidence: 99%

Interactive video asset retrieval using sketched queries

James

Collomosse

2014

Proceedings of the 11th European Conference on Visual Media Production

View full text Add to dashboard Cite

We present a new algorithm for searching video repositories using free-hand sketches. Our queries express both appearance (color, shape) and motion attributes, as well as semantic properties (object labels) enabling hybrid queries to be specified. Unlike existing sketch based video retrieval (SBVR) systems that enable hybrid queries of this form, we do not adopt a model fitting/optimization approach to match at query-time. Rather, we create an efficiently searchable index via a novel space-time descriptor that encapsulates all these properties. The real-time performance yielded by our indexing approach enables interactive refinement of search results within a relevance feedback (RF) framework; a unique contribution to SBVR. We evaluate our system over 700 sports footage clips exhibiting a variety of clutter and motion conditions, demonstrating significant accuracy and speed gains over the state of the art.

show abstract

Content-Based Image Retrieval over the Web Using Query by Sketch and Relevance Feedback

Cited by 43 publications

References 16 publications

A performance evaluation of gradient field HOG descriptor for sketch based image retrieval

A performance evaluation of gradient field HOG descriptor for sketch based image retrieval

Markov random fields for sketch based video retrieval

Interactive video asset retrieval using sketched queries

Contact Info

Product

Resources

About