Gradual transition detection using color coherence and other criteria in a video shot meta-segmentation framework

2010 IEEE Fourth International Conference on Semantic Computing

Sidiropoulos

Dimou

et al. 2010

Self Cite

Abstract-This work examines the possibility of exploiting, for the purpose of video segmentation to scenes, semantic information coming from the analysis of the visual modality. This information, in contrast to the low-level visual features typically used in previous approaches, is obtained by application of trained visual concept detectors such as those developed and evaluated as part of the TRECVID High-Level Feature Extraction Task. A large number of non-binary detectors is used for defining a high-dimensional semantic space. In this space, each shot is represented by the vector of detector confidence scores, and the similarity of two shots is evaluated by defining an appropriate shot semantic similarity measure. Evaluation of the proposed approach is performed on two test datasets, using baseline concept detectors trained on a dataset completely different from the test ones. The results show that the use of such semantic information, which we term "visual soft semantics", contributes to improved video decomposition to scenes.

Section: Overviewmentioning

confidence: 99%

Section: A Datasets and Experimental Setupmentioning

confidence: 99%

On the Use of Visual Soft Semantics for Video Temporal Decomposition to Scenes

2010 IEEE Fourth International Conference on Semantic Computing

Sidiropoulos

Dimou

et al. 2010

Self Cite

“…Thus, scene segmentation starts with application of the method of [8] for generating a decomposition S of the video to visual shots, Subsequently, as illustrated in Fig. 1, visual feature extraction is performed.…”

Section: Overview Of the Proposed Approachmentioning

confidence: 99%

On the Use of Audio Events for Improving Video Scene Segmentation

Sidiropoulos

Lecture Notes in Electrical Engineering

Kompatsiaris

et al. 2012

Self Cite

This work deals with the problem of automatic temporal segmentation of a video into elementary semantic units known as scenes. Its novelty lies in the use of high-level audio information in the form of audio events for the improvement of scene segmentation performance. More specifically, the proposed technique is built upon a recently proposed audio-visual scene segmentation approach that involves the construction of multiple scene transition graphs (STGs) that separately exploit information coming from different modalities. In the extension of the latter approach presented in this work, audio event detection results are introduced to the definition of an audio-based scene transition graph, while a visual-based scene transition graph is also defined independently. The results of these two types of STGs are subsequently combined. The application of the proposed technique to broadcast videos demonstrates the usefulness of audio events for scene segmentation.

“…Given a set of multimedia data that is possibly related with one or more of the events learned during the training stage (section 2.3.1), automatic analysis techniques, such as temporal segmentation to shots and scenes [16,19] in case of video content, are initially applied to derive a set of content segments. Each content segment is then represented with a model vector using the trained concepts detectors, and these model vectors are further projected to the discriminant subspace using the SDA projection matrix.…”

Section: Indexing Of Non-annotated Contentmentioning

confidence: 99%

Automatic event-based indexing of multimedia content using a joint content-event model

Gkalelis

Proceedings of the 2nd ACM International Workshop on Events in Multimedia

Kompatsiaris

2010

Self Cite

In this paper a joint content-event model for the automatic indexing of multimedia content with events is proposed. This model treats events as first class entities and provides a referencing mechanism for automatically linking event elements (represented using the event part of the model) with content segments (described using the content part of the model). The emphasis of the paper is on this mechanism, which uses trained concept detectors to represent content segments with model vectors, and the subclass discriminant analysis algorithm to derive a discriminant subspace facilitating the indexing of content segments with event elements. The use of this referencing mechanism for associating multimedia content with five sport events is demonstrated on the MediaMill dataset.