Simulated Testing of an Adaptive Multimedia Information Retrieval System

Abstract

[2,3]. In and implicit retrieval cycles. Our model seems to enhance re-this paper we analyse a model of implicit feedback. In our trieval results. Results are presented and discussed in the final experiment we look at the advantages the use of implicit feedsection. back can possibly give in retrieval performance, by simulating users based on the collection relevance information. The results were then analysed to investigate the assumptions on 1. INTRODUCTION which the model of implicit information is based. This paper is organised as follows: It gives a short survey With the improving capabilities of current hardware systems, of existing video retrieval systems in section 2 and presents there are ever growing possibilities to store and manipulate their inadequacies, which motivated our work. We implevideos in a digital format, leading to a growing number of mented a video retrieval system which can be used to give video archives. People build their own digital libraries from explicit and implicit relevance feedback. In section 3, we inmaterials created through digital cameras and camcorders, and troduce this system and outline the model of implicit inforuse systems such as YouTube. and Google Video2 to place mation which is built into the system. Four simulated user this material on the web. Unfortunately, this data creation studies were performed (section 4) to investigate the type of prowess is not matched by any comparable tools to organise performance improvement we might hope to gain given the and retrieve video information, use of the implicit model. In the remainder of this paper, weThere is a need to create new retrieval engines to assist the discuss our simulation results, draw conclusions and focus on user in searching and finding video scenes he/she would like our future work. Hauptmann et al. (2004) [8] developed and compared two relevance feedback model which fits best the weighting bevideo retrieval systems using visual and textual data versus a tween explicit and implicit feedback. We assume that the visual-only system as part of the Informedia project. In addiinformation given implicitly by a user can be used in video tion, they compared expert and naive users.retrieval to enhance search results. For testing this, we deHauptmann et al. (2005) [9] evaluated the system in lowveloped a retrieval system which can make use of implicit level feature extraction, semantic concept feature extraction relevance feedback. Our second objective was to provide reand searching. Their interface visualised a list of results which trieval results handling these implicit features. We developed are associated with text terms. The retrieval results are not dea first feedback model where we assumed some features as pendent on relevance feedback. Their main focus is on combeing important for implicit feedback. Based on that, we ran paring the evolution of topics and data set through the years a simulated study to get data for our objective.and measuring novice and expert users. Foley et al. (2005) [10] experimented ...

Join the Living Lab: Evaluating News Recommendations in Real-Time

“…Unfortunately though, most user studies lack of a large user base which would be required to confirm research hypotheses. Hence, addressing this shortcoming, various methodologies have been suggested such as user simulation [3] or the evaluation of systems in a playful scenario [4]. Although these approaches can be used for "fine-tuning" of algorithms [5] or evaluation in a competitive environment, the artificial nature of this experimental setup casts some doubt on to which degree these findings can be generalised.…”

Studying interaction methodologies in video retrieval

“…In the simulationbased evaluation methodology, actions that a real user may take are assumed and used to influence further retrieval results. Hopfgartner et al [11] introduced a simulation framework to evaluate adaptive multimedia retrieval systems. In order to develop a retrieval method, they employed a simulated evaluation methodology which simulated users giving implicit relevance feedback.…”

Use of Implicit Graph for Recommending Relevant Videos: A Simulated Evaluation

“…In the simulation-based evaluation methodology, actions that a real user may take are assumed and used to influence further retrieval results. Hopfgartner et al [3] introduce a simulation framework to evaluate adaptive multimedia retrieval systems. In order to develop a retrieval method, they employed a simulated evaluation methodology which simulated users giving implicit relevance feedback.…”

“…An alternative way of evaluating such user feedback is the use of simulated interactions. In such an approach, a set of possible steps are assumed when a user is performing a given task with the evaluated system [3,4,11].…”

“…Table 1 shows the probability values obtained from the user study. The simulation system, based on a system introduced by Hopfgartner et al [3], simulates a user performing one of the four tasks, using ten interactions (i.e. queries) for each task, and interacting with ten documents per query, which were the averages observed during the user experiments.…”