Fisher Kernel Temporal Variation-based Relevance Feedback for video retrieval

Mironica, Ionut; Ionescu, Bogdan; Uijlings, Jasper; Sebe, Nicu

doi:10.1016/j.cviu.2015.10.005

Cited by 18 publications

(12 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, those solutions were designed for collections far smaller than YFCC100M, which is the challenge we take in this paper. Linear models for classification, such as Linear SVM are still amongst the most frequent choices in relevance feedback applications [22,31,48] due to their simplicity, interpretability and explainability as well as the ability to produce accurate results with few annotated samples and scale to very large collections.…”

Section: Related Workmentioning

confidence: 99%

Interactive Learning for Multimedia at Large

Khan

Jónsson

Rudinac

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Interactive learning has been suggested as a key method for addressing analytic multimedia tasks arising in several domains. Until recently, however, methods to maintain interactive performance at the scale of today's media collections have not been addressed. We propose an interactive learning approach that builds on and extends the state of the art in user relevance feedback systems and high-dimensional indexing for multimedia. We report on a detailed experimental study using the ImageNet and YFCC100M collections, containing 14 million and 100 million images respectively. The proposed approach outperforms the relevant state-of-the-art approaches in terms of interactive performance, while improving suggestion relevance in some cases. In particular, even on YFCC100M, our approach requires less than 0.3 s per interaction round to generate suggestions, using a single computing core and less than 7 GB of main memory.

show abstract

Section: Related Workmentioning

confidence: 99%

Interactive Learning for Multimedia at Large

Khan

Jónsson

Rudinac

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…The interactive learning process is facilitated using a linear SVM model, proven to provide a good balance between efficiency and accuracy when classifying large datasets based on few training examples [5,9]. Based on the relevance indication provided by the user, a classifier is trained separately for the text and visual modalities and the images furthest from the hyperplane are selected.…”

Section: The Learning Processmentioning

confidence: 99%

Exquisitor

Ragnarsdóttir

Þorleiksdóttir

Khan

et al. 2019

Proceedings of the 27th ACM International Conference on Multimedia

View full text Add to dashboard Cite

In this demonstration, we present Exquisitor, a media explorer capable of learning user preferences in real-time during interactions with the 99.2 million images of YFCC100M. Exquisitor owes its efficiency to innovations in data representation, compression, and indexing. Exquisitor can complete each interaction round, including learning preferences and presenting the most relevant results, in less than 30 ms using only a single CPU core and modest RAM. In short, Exquisitor can bring large-scale interactive learning to standard desktops and laptops, and even high-end mobile devices. CCS CONCEPTS • Information systems → Multimedia and multimodal retrieval; Multimedia databases;

show abstract

“…According to Mironicȃ et al [34], relevance feedback can be incorporate in three ways. The first way is to change the query points, i.e.…”

Section: Relevance Feedbackmentioning

confidence: 99%

“…The terms used to match the query with are re-weighted according to the relevance feedback [19,24,32,42,43,52]. Another strategy is to change the Fisher representation [34] based on the relevance feedback.…”

Section: Relevance Feedbackmentioning

confidence: 99%

Improving video event retrieval by user feedback

Boer

Pingen

Knook

et al. 2017

Multimed Tools Appl

View full text Add to dashboard Cite

In content based video retrieval videos are often indexed with semantic labels (concepts) using pre-trained classifiers. These pre-trained classifiers (concept detectors), are not perfect, and thus the labels are noisy. Additionally, the amount of pre-trained classifiers is limited. Often automatic methods cannot represent the query adequately in terms of the concepts available. This problem is also apparent in the retrieval of events, such as bike trick or birthday party. Our solution is to obtain user feedback. This user feedback can be provided on two levels: concept level and video level. We introduce the method Adaptive Relevance Feedback (ARF) on video level feedback. ARF is based on the classical Rocchio relevance feedback method from Information Retrieval. Furthermore, we explore methods on concept level feedback, such as the re-weighting and Query Point Modification (QPM) methods as well as a method that changes the semantic space the concepts are represented in. Methods on both concept level and video level are evaluated on the international benchmark TRECVID Multimedia Event Detection (MED) and compared to state of the art methods. Results show that relevance feedback on both concept and video level improves performance compared to using no relevance feedback; relevance feedback on video level obtains higher performance compared to relevance feedback on concept level; our proposed ARF method on video level outperforms a state of the art k-NN method, all methods on concept level and even manually selected concepts.

show abstract

Fisher Kernel Temporal Variation-based Relevance Feedback for video retrieval

Cited by 18 publications

References 54 publications

Interactive Learning for Multimedia at Large

Interactive Learning for Multimedia at Large

Exquisitor

Improving video event retrieval by user feedback

Contact Info

Product

Resources

About