A Similarity-Based Approach for Audiovisual Document Classification Using Temporal Relation Analysis

Ibrahim, Zein Al Abidin; Ferrané, Isabelle; Joly, Philippe

doi:10.1155/2011/537372

Cited by 10 publications

(6 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The analysis of the temporal relations that exist between different segmentations has been used in several already-published works for audio-visual document structuring, event detection and audio-visual document classification [27][28][29]. The two classes to be composed…”

Section: Video Analysis Using the Defined Temporal Reation Algebramentioning

confidence: 99%

See 1 more Smart Citation

Temporal relation algebra for audiovisual content analysis

Ibrahim

Ferrané

Joly

2018

Multimed Tools Appl

Self Cite

View full text Add to dashboard Cite

The context of this work is to characterize the content and the structure of audiovisual documents by analysing the temporal relationships between basic events resulted from different segmentations of the same document. For this objective, we need to represent and reason about time. We propose a parametric representation of temporal relation between segments (points or intervals) in which the parameters are used to characterize the relationship between two non-convex intervals corresponding to two segmentations in the video analysis domain. The relationship is represented by a co-occurrences matrix noted as Temporal Relation Matrix (TRM). Each document is represented by a set of TRMs computed between each couple of segmentations of the same document using different features. The TRMs are analysed later to detect semantic events, highlight clues about the video content structure or to classify documents based on their types. For higher-level semantic events and documents' structure, we needed to apply some operations on the basic temporal relations and TRMs such as composition, disjunction, complement, intersection, etc. These operations brought to light more complex patterns; e.g. event 1 occurs at the same time of event 2 followed by event 3. In the work presented in this paper, we define a temporal relation algebra including its set of operations based on the parametric representation and TRM defined above. Several experi-mentations have been done on different audio and video documents to show the efficiency of the proposed representation and the defined operations for audiovisual content analysing.

show abstract

Section: Video Analysis Using the Defined Temporal Reation Algebramentioning

confidence: 99%

“…The distribution of votes over the clusters in each TRM was used in [29] in order to compute the similarity between audiovisual documents. Several elementary segmentations derived from dominant color, motion quantity, contrast, speakers, faces, applauses, speech, music, silence are used.…”

Section: Video Analysis Using the Defined Temporal Reation Algebramentioning

confidence: 99%

Temporal relation algebra for audiovisual content analysis

Ibrahim

Ferrané

Joly

2018

Multimed Tools Appl

Self Cite

View full text Add to dashboard Cite

show abstract

“…These features are used to train a parallel Neural Network system, which achieves an accuracy of up to 95% in distinguishing between seven video genres and sub-genres, namely football, cartoons, music, weather forecast, newscast, talk shows, and commercials. A generic approach to video categorization was discussed in [21]. Each video document is modeled by a Temporal Relation Matrix (TRM) which describes the relationship between video segments, that is temporal intervals related to the occurrence of a specific type of event.…”

Section: Related Workmentioning

confidence: 99%

Video genre categorization and representation using audio-visual information

et al. 2012

View full text Add to dashboard Cite

We propose an audio-visual approach to video genre classification using content descriptors that exploit audio, color, temporal, and contour information. Audio information is extracted at blocklevel, which has the advantage of capturing local temporal information. At the temporal structure level, we consider action content in relation to human perception. Color perception is quantified using statistics of color distribution, elementary hues, color properties, and relationships between colors. Further, we compute statistics of contour geometry and relationships. The main contribution of our work lies in harnessing the descriptive power of the combination of these descriptors in genre classification. Validation was carried out on over 91 hours of video footage encompassing 7 common video genres, yielding average precision and recall ratios of 87%−100% and 77%−100%, respectively, and an overall average correct classification of up to 97%. Also, experimental comparison as part of 1 the MediaEval 2011 benchmarking campaign demonstrated the superiority of the proposed audiovisual descriptors over other existing approaches. Finally, we discuss a 3D video browsing platform that displays movies using feature-based coordinates and thus regroups them according to genre.

show abstract

“…Two videos have the same structure if they are filmed in the same manner and thus belong to the same category. A generic attempt to represent the structure of the video is the one proposed by Ibrahim et al [5]. The representation method is based on the analysis of temporal relations between low-level features.…”

Section: Introductionmentioning

confidence: 99%

“…Algebra of temporal relation was also proposed in [6] to analyze these TRMs. Using these TRMs, the authors defined in [5] a similarity measure to classify and cluster a set of videos in one of the categories: news, soccer, TV series, documentary, TV games and movie extracts.…”

Section: Introductionmentioning

confidence: 99%

VideoToVecs: a new video representation based on deep learning techniques for video classification and clustering

2019

Self Cite

View full text Add to dashboard Cite

With the recent revolution in the field of multimedia technology, video data have become much easier and straightforward to be created, stored and transferred on a huge scale with small costs. The big amount of created data pushed the research community to delve into various study areas to aid the huge proliferation of multimedia content such as video structuring, video classification and clustering, events and objects detection, video recommendation and many other video content analysis techniques. The key success of any analysis technique relies on the audiovisual features extracted from the video. Motivated by the appearance and efficiency of deep learning techniques, we propose in this paper a new deep-learning-based features representation of videos. We depend on image-based features extracted from the sequence of frames in the video using deep learning techniques. A mapping approach named VideoToVecs is then applied to transform the extracted features into a matrix in which each row contains features of the same type. This matrix is named deep features video matrix. The efficiency of the representation is tested on 5261-video dataset for classification and clustering, and the obtained results were very promising as we will see in the paper.

show abstract

A Similarity-Based Approach for Audiovisual Document Classification Using Temporal Relation Analysis

Cited by 10 publications

References 9 publications

Temporal relation algebra for audiovisual content analysis

Temporal relation algebra for audiovisual content analysis

Video genre categorization and representation using audio-visual information

VideoToVecs: a new video representation based on deep learning techniques for video classification and clustering

Contact Info

Product

Resources

About