Temporal Video Segmentation to Scenes Using High-Level Audiovisual Features

Sidiropoulos, Panagiotis; Mezaris, Vasileios; Kompatsiaris, Ioannis; Meinedo, Hugo; Bugalho, Miguel; Trancoso, Isabel

doi:10.1109/tcsvt.2011.2138830

Cited by 139 publications

(126 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Some methods focus on scene segmentation for various types of TV programs. For example, [23] proposes a novel approach to video temporal decomposition into semantic scenes jointly exploiting lowlevel and high-level features automatically extracted from the visual and the auditory channel, where a fast scene transition graph (STG) approximation and a generalized STG-based technique are proposed for multimodal scene segmentation. [31] focuses on grouping video content into semantic segments and classifying semantic scenes into different types based on the temporal constraint of video content and visual similarity between shot activities.…”

Section: Related Workmentioning

confidence: 99%

Content-based unsupervised segmentation of recurrent TV programs using grammatical inference

Vallet

Carrive

et al. 2017

Multimed Tools Appl

View full text Add to dashboard Cite

TV program segmentation raised as a major topic in the last decade for the task of high quality indexing of multimedia content. Earlier studies of TV program segmentation are either highly supervised (e.g., event detection) or too specific to a certain type of program (e.g., cluster-based methods), which is not practically usable for indexing tasks because of the lack of generality of programs types. In this paper, we address the problem of unsupervised TV program segmentation by leveraging grammatical inference, i.e., discovering a common structural model shared by a collection of episodes of a recurrent TV program by finding an optimal alignment of structural elements across episodes. Structural elements referring to a video segment with a particular syntactic meaning with respect to the video structure. The use of symbolic representation of structural elements makes grammatical inference feasible to be applied on TV program modeling, and makes TV program segmentation possible to rely on only minimal domain knowledge. The proposed approach is operated in two phases. The first phase aims at obtaining a symbolic representation of each episode, where the elements relevant to the structure are discovered based on recurrence mining. The second phase is that of grammatical inference from the symbolic representation of episodes. We investigate two inference techniques, one based on multiple sequence alignment and one relying on uniform resampling, to infer structural grammars for TV programs. A model of the structure is derived from the structural grammars and used to predict the structure of new episodes. Comparative evaluation on two gramBingqing Qu

show abstract

Section: Related Workmentioning

confidence: 99%

Content-based unsupervised segmentation of recurrent TV programs using grammatical inference

Vallet

Carrive

et al. 2017

Multimed Tools Appl

View full text Add to dashboard Cite

show abstract

“…2), adaptive thresholds (thresholds depend on the statistics of the visual features used), B-splines fittings [30], support vector machines (SVM) [31] and K-means clustering [11]. The detection accuracy of SBD methods is improved by combining several visual features [32].…”

Section: Shot Boundary Detectionmentioning

confidence: 99%

Towards key-frame extraction methods for 3D video: a review

Ferreira

Cruz

Assunção

2016

J Image Video Proc.

View full text Add to dashboard Cite

The increasing rate of creation and use of 3D video content leads to a pressing need for methods capable of lowering the cost of 3D video searching, browsing and indexing operations, with improved content selection performance. Video summarisation methods specifically tailored for 3D video content fulfil these requirements. This paper presents a review of the state-of-the-art of a crucial component of 3D video summarisation algorithms: the key-frame extraction methods. The methods reviewed cover 3D video key-frame extraction as well as shot boundary detection methods specific for use in 3D video. The performance metrics used to evaluate the key-frame extraction methods and the summaries derived from those key-frames are presented and discussed. The applications of these methods are also presented and discussed, followed by an exposition about current research challenges on 3D video summarisation methods.Keywords: 3D key-frames extraction, 3D video summarisation, Shot boundary detection ReviewIn the last years, new features have been implemented in video applications and terminal equipments due to users demand, who are always seeking for new viewing experiences more interactive and immersive, such as those provided by 3D video. This new visual experience is created by depth information that is part of 3D video and absent in classic 2D video. The inclusion of depth information in video signals is not a recent innovation, but the interest in this type of content and aspects related to it, such as acquisition, analysis, coding, transmission and visualisation, have been increasing recently [1,2]. Lately, 3D video has been attracting attention from industry, namely content producers, equipment providers, distributors and from the research community mostly on account of the improvements in Quality of Experience that it provides to viewers [3], as well as due to the new business opportunities presented by this emerging multimedia format. In the past, video repositories were relatively small so that indexing and retrieval operations were easy to perform. More recently, the massification of 3D video and its applications have resulted in the generation of huge amounts of data, increasing the need for methods that can efficiently index, search, browse and summarise the relevant information with minimum human intervention. Furthermore, 3D video description and management is also required to enable quick presentation of the most important information in a user-friendly manner [4,5]. Video summarisation is a video-content representation method that can fulfil these requirements. In contrast to summarisation of 2D video, which has been the subject of a significant amount of research, 3D video summarisation is still a relatively unexplored research problem which deserves more attention.A video summary is a short version of a full-length video that preserves the essential visual and semantic information of the original unabridged content. In the video summarisation process, a subset of key-frames or a set of shorter video...

show abstract

“…Most recent techniques, e.g. [10], [12], further exploit higher-level information such as visual concept and audio event detection results in order to come to a more accurate extraction of the videos' structural semantics. Specifically, in [10] the possibility of exploiting, for the purpose of video segmentation to scenes, semantic information coming from the analysis of the visual modality, was examined.…”

Section: Retrievalmentioning

confidence: 99%

Improving Interactive Video Retrieval by Exploiting Automatically-Extracted Video Structural Semantics

Mezaris

Sidiropoulos

Kompatsiaris

2011

2011 IEEE Fifth International Conference on Semantic Computing

View full text Add to dashboard Cite

Abstract-In this work the contribution of automaticallyextracted (thus, imperfect) video structural semantics towards improving interactive video retrieval is examined. First, the automatic extraction of video structural semantics, i.e. the decomposition of the video into scenes that correspond to the different sub-stories or high-level events, is performed. Then, these are introduced to the interactive video retrieval paradigm. Finally, their potential contribution is experimentally evaluated. To this end, different members of a family of scene segmentation algorithms are applied to an extensive professional video collection coming from the TRECVID benchmarking activity; subsequently, a large number of user interactions with a retrieval system that exploits these structural semantics is simulated. The experimental results document the contribution of state-of-the-art automatically-extracted video structural semantics to the efficient and effective interactive video retrieval.

show abstract

Temporal Video Segmentation to Scenes Using High-Level Audiovisual Features

Cited by 139 publications

References 31 publications

Content-based unsupervised segmentation of recurrent TV programs using grammatical inference

Content-based unsupervised segmentation of recurrent TV programs using grammatical inference

Towards key-frame extraction methods for 3D video: a review

Improving Interactive Video Retrieval by Exploiting Automatically-Extracted Video Structural Semantics

Contact Info

Product

Resources

About