Automatic fine-grained hyperlinking of videos within a closed collection using scene segmentation

Abstract

This paper introduces a framework for establishing links between related media fragments within a collection of videos. A set of analysis techniques is applied for extracting information from different types of data. Visual-based shot and scene segmentation is performed for defining media fragments at different granularity levels, while visual cues are detected from keyframes of the video via concept detection and optical character recognition (OCR). Keyword extraction is applied on textual data such as the output of OCR, subtitles and metadata. This set of results is used for the automatic identification and linking of related media fragments. The proposed framework exhibited competitive performance in the Video Hyperlinking sub-task of MediaEval 2013, indicating that video scene segmentation can provide more meaningful segments, compared to other decomposition methods, for hyperlinking purposes

Capturing News Stories Once, Retelling a Thousand Ways

“…Something similar occurs in [1] where some small video fragments are hyperlinked to others based on some visual or topical similarity. Even if such a task can be tackled using multimodal analysis techniques, applications at early stages of the news consumption focus mostly in finding content as similar as possible to the original.…”

Video hyperlinking

“…This brings new challenges in automatically processing the visual content and understanding the information it conveys at different granularities, in processing associated audio and textual information, and in intelligently exploiting all these analysis results for creating meaningful video hyperlinks [4]. It also raises important questions concerning the granularities that are most appropriate for decomposing and linking the video content [3].…”

A Study on Multimodal Video Hyperlinking with Visual Aggregation

“…Video hyperlinking systems usually start from a set of anchors that define entry points of interest in collections of long videos and are required to provide, for each anchor, relevant targets within the collection. This task is usually implemented as a two-step process, first starting from a segmentation of the long videos into small segments, then selecting relevant segments for a given anchor [4,5]. This last step is cast as a video retrieval task relying on video segment comparison, where various multimodal solutions have been proposed.…”