Hierarchical structuring of video previews by Leading-Cluster-Analysis

Abstract

Clustering of shots is frequently used for accessing video data and enabling quick grasping of the associated content. In this work we first group video shots by a classic hierarchical algorithm, where shot content is described by a codebook of visual words and different codebooks are compared by a suitable measure of distortion. To deal with the high number of levels in a hierarchical tree, a novel procedure of Leading-Cluster-Analysis is then proposed to extract a reduced set of hierarchically arranged previews. The depth of the obtained structure is driven both from the nature of the visual content information, and by the user needs, who can navigate the obtained video previews at various levels of representation. The effectiveness of the proposed method is demonstrated by extensive tests and comparisons carried out on a large collection of video data.

Interactive Film Recombination

“…To let the video recombination process take advantage of the already existent narrative structure of the baseline movie, the video processing unit aggregates groups of adjacent shots using the shots' low-level features and their temporal relations, forming so called Logical Story Units (or LSUs) [4], which model the baseline movie scenes (see Section 5.1). By joining the LSU segmentation information and the semantic description of the shots therein, as described in Section 5.2, a new set of models is obtained that are referred to as Semantic Story Units (or SSUs) [28], which are basically Markov chains.…”

“…At the atomic level, the baseline video is first decomposed into shots using a traditional shot-cut detector [11], which typically works by analyzing the variations of the statistical color intensity distributions of the video frames. Sequences of shots conveying a common concept in the context of the story are then grouped into LSUs [4]. To do so, the shots are first clustered into nodes using both a measure of visual similarity and temporal distance.…”

Generating story variants with constrained video recombination

“…Then, a shot similarity measure is defined by averaging the distortion increase caused by representing each shot using the codebook of the other. Last, the final shot visual clusters are obtained through hierarchical clustering as in [2].…”

Interactive storytelling via video content recombination

“…At the beginning of the hierarchical clustering each shot belongs to a different cluster, then the algorithm iteratively merges the two most similar clusters, where similarity between clusters is given by the average similarity between all shots belonging to the two clusters. The final hierarchical grouping is obtained by exploiting the properties of the associated dendrogram tree and the codebook distortions related to different branches of the dendrogram (see [1] for details).…”