Audio-Visual Fusion for Detecting Violent Scenes in Videos

Γιαννακόπουλος, Θεόδωρος; Makris, Alexandros; Kosmopoulos, Dimitrios; Perantonis, Stavros; Theodoridis, Sergios

doi:10.1007/978-3-642-12842-4_13

Cited by 78 publications

(45 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…More recently, Yan et al [17] have developed a Multi-task Learning approach for head-pose estimation in a multicamera environment under target motion. Giannakopoulos et al [5], in an attempt to extend their approach based solely on audio cues [7], propose to use a multimodal two-stage approach based on k nearest neighbors (k-NN). In the first step, they perform audio and visual analysis of segments of one second duration.…”

Section: Related Workmentioning

confidence: 99%

“…Recently, solutions using mid-level feature representations have gained popularity. These solutions shifted away not only from the traditional approaches which represented videos using low-level features (e.g., [4,5]) but also from the use of state-of-theart detectors designed to identify high-level semantic concepts (e.g., "a killing spree"). The earlier solutions could not carry enough semantic information, and the latter ones have not reached a sufficient level of maturity.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Breaking down violence detection: Combining divide-et-impera and coarse-to-fine strategies

2016

View full text Add to dashboard Cite

Breaking down violence detection: combining divide-et-impera and coarse-to-fine strategies. Neurocomputing, 208, pp. 225-237. (doi:10.1016Neurocomputing, 208, pp. 225-237. (doi:10. /j.neucom.2016 This is the author's final accepted version.There may be differences between this version and the published version. You are advised to consult the publisher's version if you wish to cite from it.http://eprints.gla.ac.uk/119160/

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Breaking down violence detection: Combining divide-et-impera and coarse-to-fine strategies

2016

View full text Add to dashboard Cite

show abstract

“…This causes researchers difficulty in terms of working on a common ground [1]. Some of the violence interpretations include violent actions by humans where there is blood [2], scenes containing gunshots, fights and explosions [3], person to person harmful acts like threatening and physical harm [4], and fighting scenes regardless of number of individuals involved and context [5,6]. These different interpretations lead to different techniques for VSD, which makes it difficult to conduct a comparative study.…”

Section: Introductionmentioning

confidence: 99%

“…Many researchers, on the other hand, have been interested in combining both auditory and visual modalities. The combined use of audio (e.g., chroma, spectrogram and Mel-Frequency Cepstral Coefficients (MFCC)) and visual features (e.g., motion based variance, motion of people and average motion) produced some good results [4]. In [9], the authors performed a modified probabilistic Latent Semantic Analysis (pLSA) based violence detection from audio cues and visual information by exploiting different concepts (including explosion, motion, blood and flame etc.).…”

Section: Introductionmentioning

confidence: 99%

Violent Scene Detection Using a Super Descriptor Tensor Decomposition

Khokher

Bouzerdoum

Phung

2015

2015 International Conference on Digital Image Computing: Techniques and Applications (DICTA)

View full text Add to dashboard Cite

This article presents a new method for violent scene detection using super descriptor tensor decomposition. Multi-modal local features comprising auditory and visual features are extracted from Mel-frequency cepstral coefficients (including first and second order derivatives) and refined dense trajectories. There is usually a large number of dense trajectories extracted from a video sequence; some of these trajectories are unnecessary and can affect the accuracy. We propose to refine the dense trajectories by selecting only discriminative trajectories in the region of interest. Visual descriptors consisting of oriented gradient and motion boundary histograms are computed along the refined dense trajectories. In traditional bag-of-visual-words techniques, the feature descriptors are concatenated to form a single large feature vector for classification. This destroys the spatio-Temporal interactions among features extracted from multi-modal data. To address this problem, a super descriptor tensor decomposition is proposed. The extracted feature descriptors are first encoded using super descriptor vector method. Then the encoded features are arranged as tensors so as to retain the spatioTemporal structure of the features. To obtain a compact set of features for classification, the TUCKER-3 decomposition is applied to the super descriptor tensors, followed by feature selection using Fisher feature ranking. The obtained features are fed to a support vector machine classifier. Experimental evaluation is performed on violence detection benchmark dataset, MediaEval VSD2014. The proposed method outperforms most of the state-of-The-Art methods, achieving MAP2014 scores of 60.2% and 67.8% on two subsets of the dataset. Abstract-This article presents a new method for violent scene detection using super descriptor tensor decomposition. Multi-modal local features comprising auditory and visual features are extracted from Mel-frequency cepstral coefficients (including first and second order derivatives) and refined dense trajectories. There is usually a large number of dense trajectories extracted from a video sequence; some of these trajectories are unnecessary and can affect the accuracy. We propose to refine the dense trajectories by selecting only discriminative trajectories in the region of interest. Visual descriptors consisting of oriented gradient and motion boundary histograms are computed along the refined dense trajectories. In traditional bag-of-visual-words techniques, the feature descriptors are concatenated to form a single large feature vector for classification. This destroys the spatio-temporal interactions among features extracted from multi-modal data. To address this problem, a super descriptor tensor decomposition is proposed. The extracted feature descriptors are first encoded using super descriptor vector method. Then the encoded features are arranged as tensors so as to retain the spatio-temporal structure of the features. To obtain a compact set of features for classification, the TUCKER-3 de...

show abstract

“…Três dos descritores de áudio mais comumente utilizados são MFCC (Mel-Frequency Cepstrum Coefficients), LPCC (Linear Prediction Cepstral Coefficients) e Chroma (Giannakopoulos et al, 2010).…”

Section: Segmentação Em Cenas Com Descritores Sonorosunclassified

Detecção de cenas em segmentos semanticamente complexos

Lopes¹

View full text Add to dashboard Cite

Dedico esse trabalho a meus pais e à minha noiva, que sempre me apoiaram e ajudaram em todos os momentos. AgradecimentosAgradeço em primeiro lugar a Deus, por ter me iluminado durante todo o desenvolvimento do trabalho, dando a paciência e a inspiração necessária para sua realização.Agradeço também, a meu orientador, pelos infindáveis conselhos e por sua orientação sempre tão pertinente.Agradeço aos professores das matérias realizadas no mestrado, que certamente contribuíram beneficamente para a realização dessa pesquisa.Agradeço aos colegas e amigos do laboratório de pesquisa, que sempre me apoiaram e me deram forças nos momentos de desânimo.Agradeço ao CNPq pelo auxílio financeiro, processo n°134245/2011-3. Agradeço à FAPESP pelo auxílio financeiro, processo n°2011/05238-0, Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP). "As opiniões, hipóteses e conclusões ou recomendações expressas neste material são de responsabilidade do(s) autor(es) e não necessariamente refletem a visão da FAPESP". Resumo Diversas áreas da Computação (Personalização e Adaptação de Conteúdo, Recuperação de Informação, entre outras) se beneficiam da segmentação de vídeo em unidades menores de informação. A literatura apresenta diversos métodos e técnicas cujo objetivo é identificar essas unidades. Uma limitação é que tais técnicas não tratam o problema da detecção de cenas em segmentos semanticamente complexos, definidos como trechos de vídeo que apresentam mais de um assunto ou tema, e cuja semân-tica latente dificilmente pode ser determinada utilizando-se somente uma única mídia. Esses segmentos são muito relevantes, pois estão presentes em diversos domínios de vídeo, tais como filmes, noticiários e mesmo comerciais.A presente Dissertação de Mestrado propõe uma técnica de segmentação de vídeo capaz de identificar cenas em segmentos semanticamente complexos. Para isso utiliza a semântica latente alcançada com o uso de Bag of Visual Words para agrupar os segmentos de um vídeo. O agrupamento é baseado em multimodalidade, analisando-se características visuais e sonoras de cada vídeo e combinando-se os resultados por meio da estratégia fusão tardia. O presente trabalho demonstra a viabilidade técnica em reconhecer cenas em segmentos semanticamente complexos. The literature reports lots of techniques and methods, whose goal is to identify these units. One of these techniques' limitations is that they don't handle scene detection in semantically complex segments, which are defined as video snippets that present more than one subject or theme, whose latent semantics can hardly be determined using only one media. Those segments are very relevant, since they are present in multiple video domains as movies, news and even television commercials. This Master's dissertation proposes a video scene segmentation technique able to detect scenes in semantically complex segments. In order to achieve this goal it uses latent semantics extracted by the Bag of Visual Words to group a video segments. This grouping process is based on multimodalit...

show abstract

Audio-Visual Fusion for Detecting Violent Scenes in Videos

Cited by 78 publications

References 10 publications

Breaking down violence detection: Combining divide-et-impera and coarse-to-fine strategies

Breaking down violence detection: Combining divide-et-impera and coarse-to-fine strategies

Violent Scene Detection Using a Super Descriptor Tensor Decomposition

Detecção de cenas em segmentos semanticamente complexos

Contact Info

Product

Resources

About