Automatic Story Segmentation for TV News Video Using Multiple Modalities

Abstract

While video content is often stored in rather large files or broadcasted in continuous streams, users are often interested in retrieving only a particular passage on a topic of interest to them. It is, therefore, necessary to split video documents or streams into shorter segments corresponding to appropriate retrieval units. We propose here a method for the automatic segmentation of TV news videos into stories. A-multiple-descriptor based segmentation approach is proposed. The selected multimodal features are complementary and give good insights about story boundaries. Once extracted, these features are expanded with a local temporal context and combined by an early fusion process. The story boundaries are then predicted using machine learning techniques. We investigate the system by experiments conducted using TRECVID 2003 data and protocol of the story boundary detection task, and we show that the proposed approach outperforms the state-of-the-art methods while requiring a very small amount of manual annotation.

Automatic topics segmentation for TV news video using prior knowledge

“…For example, Dumont and Quénot presented a system based on multimodal features extraction [7]. The approach combines audio features as silence segments and visual descriptors like anchors or logos.…”

“…In these experiments, the approaches described in [7,9,20] were selected for comparison according to two criteria. First, we chose methods based on the same assumption to segment news programs: anchorperson shots detection as the starting point for detecting news topics.…”

“…The second criterion is that all approaches were experimented on the same standard TRECVID 2003 dataset. To reach an accurate comparison with other works, we used same metrics as defined in [7]. So, we evaluated the news segmentation performance of stand-alone and rewire modes using the precision, recall and F1 metrics.…”

Content-based unsupervised segmentation of recurrent TV programs using grammatical inference

“…A temporal structural model is used in [29] to identify the different news stories in broadcast, while [8] uses machine-learning-based techniques to classify the shots of news video into predefined categories, e.g., anchor, interview, forcast. [10] proposes a method for the automatic segmentation of TV news videos into stories, where a temporal context and machine learning methods are used to perform the story boundaries detection from multimodal features. There exists some work focusing only on speech recognition of TV programs using Deep Neural Networks (DNN), for example, [24] uses generalized discriminant analysis for acoustic feature extraction and [11] represents acoustic features by an i-vector before adopting DNN techniques.…”

“…The most traditional way to realize automatic TV program segmentation is either classification strategies [10,27] or event detection approaches [4,17], which are mostly supervised approaches. Unsupervised approaches for program segmentation were also addressed recently, where audiovisual consistency [5] and clustering-based methods [15] are considered.…”