A monocular system for person tracking: Implementation and testing

Lecture Notes in Computer Science

Nikolaidis

et al. 2015

Self Cite

Abstract. Multimodal clustering/diarization tries to answer the question "who spoke when" by using audio and visual information. Diarization consists of two steps, at first segmentation of the audio information and detection of the speech segments and then clustering of the speech segments to group the speakers. This task has been mainly studied on audiovisual data from meetings, news broadcasts or talk shows. In this paper, we use visual information to aid speaker clustering. We tested the proposed method in three full length movies, i.e. a scenario much more difficult than the ones used so far, where there is no certainty that speech segments and video appearances of actors will always overlap. The results proved that the visual information can improve the speaker clustering accuracy and hence the diarization process.

Section: Resultsmentioning

confidence: 99%

“…Faces are detected using [11] and tracked in the video channel or channels (in the case of 3D videos) of a movie using the algorithm proposed in [14]. In more detail, each detected face is tracked for K frames.…”

Section: Video Processingmentioning

confidence: 99%

Multimodal Speaker Diarization Utilizing Face Clustering Information

Kapsouras

Lecture Notes in Computer Science

Nikolaidis

et al. 2015

Self Cite

“…The proposed method operates on grayscale facial moving regions. Face detection and tracking [11], [12] techniques are used to find such regions in a video. After determining the facial Regions of Interest (ROIs) in each facial video sequence, we find the union R = {∪R k , k = 1, .…”

Section: Proposed V-vad Methodsmentioning

confidence: 99%

“…The proposed method is formed by three processing steps. In the first step, a face detection technique [11] is applied to a video frame, in order to determine the facial Region of Interest (ROI), which is subsequently tracked over time [12], in order for a facial ROI trajectory of the person under investigation to be created. Such videos are noted as facial moving regions hereafter.…”

Section: Introductionmentioning

confidence: 99%

Visual Voice Activity Detection in the Wild

Patrona

Iosifidis

et al. 2016

IEEE Trans. Multimedia

Self Cite

The Visual Voice Activity Detection (V-VAD) problem in unconstrained environments is investigated in this paper. A novel method for V-VAD in the wild, exploiting local shape and motion information appearing at spatiotemporal locations of interest for facial video description and the Bag of Words (BoW) model for facial video representation, is proposed. Facial video classification is subsequently performed using state-of-theart classification algorithms. Experimental results on one publicly available V-VAD data set denote the effectiveness of the proposed method, since it achieves better generalization performance in unseen users, when compared with recently proposed state-ofthe-art methods. Additional results on a new, unconstrained, data set provide evidence that the proposed method can be effective even in such cases in which any other existing method fails.

“…The proposed method operates on grayscale facial region videos, extracted by applying face detection and tracking [13,14] techniques. After determining the facial Regions of Interest (ROIs) in each video sequence, the facial ROIS are cropped and the resulting facial images are sized to fixed size of H × W , thus producing the facial videos.…”

Section: Proposed Methodsmentioning

confidence: 99%

Visual voice activity detection based on spatiotemporal information and bag of words

Patrona

Iosifidis

2015 IEEE International Conference on Image Processing (ICIP)

et al. 2015

Self Cite

A novel method for Visual Voice Activity Detection (V-VAD) that exploits local shape and motion information appearing at spatiotemporal locations of interest for facial region video description and the Bag of Words (BoW) model for facial region video representation is proposed in this paper. Facial region video classification is subsequently performed based on Single-hidden Layer Feedforward Neural (SLFN) network trained by applying the recently proposed kernel Extreme Learning Machine (kELM) algorithm on training facial videos depicting talking and non-talking persons. Experimental results on two publicly available V-VAD data sets, denote the effectiveness of the proposed method, since better generalization performance in unseen users is achieved, compared to recently proposed state-of-the-art methods.