Multimodal genre classification of TV programs and YouTube videos

Ekenel, Hazım Kemal; Semela, Tomas

doi:10.1007/s11042-011-0923-x

Cited by 28 publications

(31 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…4 In contrast to the previous audio sets, the RAI video dataset allows for the development and comparison of approaches using different modalities. Recent works demonstrate a tendency toward multimodal approaches [11,37,38]. The results achieved by our approach indicate the outstanding performance of the selected audio features in terms of F1-score of 95.80% despite the use of single modality only.…”

Section: Comparison To Related Work On Media Classificationmentioning

confidence: 68%

“…The results achieved by our approach indicate the outstanding performance of the selected audio features in terms of F1-score of 95.80% despite the use of single modality only. Additionally, our approach demonstrates strong competitiveness to the top reported performance by Ekenel and Semela [11]. In addition to some acoustic features, Ekenel and Semena consider visual, structural, and cognitive features.…”

Section: Comparison To Related Work On Media Classificationmentioning

confidence: 76%

“…Current research on audio and video classifications focuses mostly on the development of new features and classification methods [11,26,27,30,38]. Few works exploit feature selection in the context of genre classifications [9,12,22,50,60], musical instrument classification [8,51], and emotion/mood classification [43,59], commonly using a single dataset only.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Unsupervised group feature selection for media classification

Zaharieva

Breiteneder

Hudec

2017

Int J Multimed Info Retr

View full text Add to dashboard Cite

The selection of an appropriate feature set is crucial for the efficient analysis of any media collection. In general, feature selection strongly depends on the data and commonly requires expert knowledge and previous experiments in related application scenarios. Current unsupervised feature selection methods usually ignore existing relationships among components of multi-dimensional features (group features) and operate on single feature components. In most applications, features carry little semantics. Thus, it is less relevant if a feature set consists of complete features or a selection of single feature components. However, in some domains, such as content-based audio retrieval, features are designed in a way that they, as a whole, have considerable semantic meaning. The disruption of a group feature in such application scenarios impedes the interpretability of the results. In this paper, we propose an unsupervised group feature selection algorithm based on canonical correlation analysis (CCA). Experiments with different audio and video classification scenarios demonstrate the outstanding performance of the proposed approach and its robustness across different datasets.

show abstract

Section: Comparison To Related Work On Media Classificationmentioning

confidence: 68%

Section: Comparison To Related Work On Media Classificationmentioning

confidence: 76%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Unsupervised group feature selection for media classification

Zaharieva

Breiteneder

Hudec

2017

Int J Multimed Info Retr

View full text Add to dashboard Cite

show abstract

“…In Ref. 12, for example, the authors present an automatic video genre classification system for classifying the types of TV programs, based on several low level audio-visual features, such as, color, texture, signal energy and mel-frequency cepstral coefficients, as well as cognitive and structural information derived from faces and video shots. The proposed system was evaluated using TV programs from Italian and French TV channels, achieving an overall accuracy as high as 94.5%.…”

Section: Introductionmentioning

confidence: 99%

A unified approach to content-based indexing and retrieval of digital videos from television archives

Souza¹,

Pádua

Nunes³

et al. 2014

AIR

View full text Add to dashboard Cite

This work addresses the development of a unified approach to content-based indexing and retrieval of digital videos from television archives. The proposed approach has been designed to deal with arbitrary television genres, making it suitable for various applications. To achieve this goal, the main steps of a content-based video retrieval system are addressed in this work, namely: video segmentation, key-frame extraction, content-based video indexing and the video retrieval operation itself. Video segmentation is addressed as a typical TV broadcast structuring problem, which consists in automatically determining the boundaries of each broadcasted program (like movies, news, among others) and inter-program (for instance, commercials). Specifically, to segment the videos, Electronic Program Guide (EPG) metadata is combined with the detection of two special cues, namely, audio cuts (silence) and dark monochrome frames. On the other hand, a color histogram-based approach performs key-frame extraction. Video indexing and retrieval are accomplished by using hashing and k-d tree methods, while visual signatures containing color, shape and texture information are estimated for the key-frames, by using image and frequency domain techniques. Experimental results with the dataset of a multimedia information system especially developed for managing television broadcast archives demonstrate that our approach works efficiently, retrieving videos in 0.16 seconds on average and achieving recall, precision and F1 measure values, as high as 0.76, 0.97 and 0.86 respectively.

show abstract

“…It is not surprising then that conventional uni-modal human-machine interactions lag in performance, robustness and naturalness when compared with human-human interactions. Recently, there has been increasing research interest in jointly processing information in multiple modalities and mimicking human-human multimodal interactions [2,4,5,9,13,14,16,18,19,21,22]. For example, human speech production and perception are bimodal in nature: visual cues have a broad influence on perceived auditory stimuli [17].…”

mentioning

confidence: 99%

Multimodal joint information processing in human machine interaction: recent advances

Xie

Deng

Cox

2013

Multimed Tools Appl

View full text Add to dashboard Cite

Interaction between humans is a multimodal process in nature, which integrates the different modalities of vision, hearing, gesture and touch. It is not surprising then that conventional uni-modal human-machine interactions lag in performance, robustness and naturalness when compared with human-human interactions. Recently, there has been increasing research interest in jointly processing information in multiple modalities and mimicking human-human multimodal interactions [2,4,5,9,13,14,16,18,19,21,22]. For example, human speech production and perception are bimodal in nature: visual cues have a broad influence on perceived auditory stimuli [17]. The latest research in speech processing has shown that integration of both auditory and visual information, i.e., acoustic speech combined with facial and lip motions, achieves significant performance improvement in tasks such as speech recognition [3] and emotion recognition [16]. However, information from difference modalities can be difficult to integrate: the representations of different signals are heterogeneous and the signals are often asynchronous and loosely-coupled. Therefore, finding more effective ways to integrate and jointly process information from different modalities is essential for the success of such multimodal human-machine interactive systems. Meanwhile, multimodal applications are becoming increasingly important, especially for mobile computing and digital entertainment scenarios, such as natural user interfaces (NUI) on smartphones and motion sensing gaming.This special issue aims to bring together work by researchers and technologists engaged in the development of multimodal technologies for information processing, emerging multimedia applications and user-centric human computer interaction. We received more than L. Xie ( )

show abstract

Multimodal genre classification of TV programs and YouTube videos

Cited by 28 publications

References 16 publications

Unsupervised group feature selection for media classification

Unsupervised group feature selection for media classification

A unified approach to content-based indexing and retrieval of digital videos from television archives

Multimodal joint information processing in human machine interaction: recent advances

Contact Info

Product

Resources

About