Overview of VideoCLEF 2009: New Perspectives on Speech-Based Multimedia Content Enrichment

Abstract

VideoCLEF 2009 offered three tasks related to enriching video content for improved multimedia access in a multilingual environment. For each task, video data (Dutch-language television, predominantly documentaries) accompanied by speech recognition transcripts were provided. The Subject Classification Task involved automatic tagging of videos with subject theme labels. The best performance was achieved by approaching subject tagging as an information retrieval task and using both speech recognition transcripts and archival metadata. Alternatively, classifiers were trained using either the training data provided or data collected from Wikipedia or via general Web search. The Affect Task involved detecting narrative peaks, defined as points where viewers perceive heightened dramatic tension. The task was carried out on the "Beeldenstorm" collection containing 45 short-form documentaries on the visual arts. The best runs exploited affective vocabulary and audience directed speech. Other approaches included using topic changes, elevated speaking pitch, increased speaking intensity and radical visual changes. The Linking Task, also called "Finding Related Resources Across Languages," involved linking video to material on the same subject in a different language. Participants were provided with a list of multimedia anchors (short video segments) in the Dutch-language "Beeldenstorm" collection and were expected to return target pages drawn from English-language Wikipedia. The best performing methods used the transcript of the speech spoken during the multimedia anchor to build a query to search an index of the Dutchlanguage Wikipedia. The Dutch Wikipedia pages returned were used to identify related English pages. Participants also experimented with pseudo-relevance feedback, query translation and methods that targeted proper names.

Context & Semantics in News & Web Search

“…For the first challenge, two related tasks have been considered at the CLEF evaluation campaigns. VideoCLEF'09 [120] featured a task aimed at linking video content to related resources across languages. This task was framed as a known-item-task, where noisy ASR for Dutch was used to produce links to target English Wikipedia articles.…”

“…The first step of our approach to real-time entity linking generates link candidates and rank these candidates initially. The second step is aimed at improving precision using a learning to rerank approach, that was effective on similar tasks [120,141,149]. For link candidates many ranking criteria are in play, making learning to rerank particularly appropriate.…”

Automatic Summarization

“…One of the major goals of automatic classification research is to reproduce the classification that would be generated by a human, given a specific classification scheme. For example, in [213], classes are drawn from the Media Topic Taxonomy of the International Press Telecommunications Council, 4 in [159] classes are drawn from the thesaurus used by archivists at the Netherlands Institute for Sound and Vision, a large audio-video archive, and in [161] classes are drawn from the set of tags assigned by users to video on blip.tv, a video sharing platform. Note that some forms of classification that are potentially useful for SCR perform their categorization of speech media based on characteristics not directly related to topic, such as genre in video [267] or dialogue acts in spoken conversation (i.e., statement, question, apology) [259].…”

Exploiting Speech Recognition Transcripts for Narrative Peak Detection in Short-Form Documentaries

“…Henk van Os is highly acclaimed and appreciated in the Netherlands, where he has established his ability to appeal to a broad audience. 4 Constraining the corpus to contain episodes from Beeldenstorm limits the spoken content to a single speaker speaking within the style of a single documentary series. This limitation is imposed in order to help control effects that could be introduced by variability in style or skill.…”

“…Up to five different runs (i.e., system outputs created according to different experimental conditions) could be submitted. Further details about the data set and the Affect Detection task for VideoCLEF 2009 can be found in the track overview paper [4]. Participants were provided with additional resources accompanying the test data, including transcripts generated by an automatic speech recognition system [3].…”