Hierarchical On-line Appearance-Based Tracking for 3D head pose, eyebrows, lips, eyelids and irises

Orozco, Javier; Rudovic, Ognjen; Gonzílez, Jordi; Pantić, Maja

doi:10.1016/j.imavis.2013.02.001

Cited by 39 publications

(22 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• Facial point tracking, image registration and Region of Interest (ROI) extraction • Appearance and shape features computation • Classification First, facial characteristic landmarks on the speaker's face are tracked throughout each video of an utterance, using the Appearance-Based Tracker [35]. Only the points corresponding to the lower face region are used in further processing.…”

Section: Overview Of the Proposed Methodsmentioning

confidence: 99%

Discrimination Between Native and Non-Native Speech Using Visual Features Only

Georgakis

Petridis

Pantić

2016

IEEE Trans. Cybern.

Self Cite

View full text Add to dashboard Cite

Abstract-Accent is a soft biometric trait that can be inferred from pronunciation and articulation patterns characterising the speaking style of an individual. Past research has addressed the task of classifying accent, as belonging to a native language speaker or a foreign language speaker, by means of the audio modality only. However, features extracted from the visual stream of speech have been successfully used to extend or substitute audio-only approaches that target speech or language recognition. Motivated by these findings, we investigate to what extent temporal visual speech dynamics attributed to accent can be modelled and identified when the audio stream is missing or noisy, and the speech content is unknown. We present here a fully automated approach to discriminating native from non-native English speech, based exclusively on visual cues. A systematic evaluation of various appearance and shape features for the target problem is conducted, with the former consistently yielding superior performance. Subject-independent cross-validation experiments are conducted on mobile phone recordings of continuous speech and isolated word utterances spoken by 56 subjects from the challenging MOBIO Database. High performance is achieved on a text-dependent protocol, with the best score of 76.5% yielded by fusion of five HMMs trained on appearance features. Our framework is also efficient even when tested on examples of speech unseen in the training phase, although performing less accurately compared to the text-dependent case.

show abstract

Section: Overview Of the Proposed Methodsmentioning

confidence: 99%

Discrimination Between Native and Non-Native Speech Using Visual Features Only

Georgakis

Petridis

Pantić

2016

IEEE Trans. Cybern.

Self Cite

View full text Add to dashboard Cite

show abstract

“…We use a portion of the database running approximately 85 minutes, which has been annotated for the emotion dimensions at hand by 5 raters, from which we use the averaged annotation 2 . For extracting facial expression features, we employ an Active Appearance Model (AAM) based tracker [9], designed for simultaneous tracking of 3D head pose, lips, eyebrows, eyelids and irises in video sequences. For each video frame, we obtain 113 2D-points, resulting in an 226 dimensional feature vector.…”

Section: Data and Feature Extractionmentioning

confidence: 99%

Correlated-spaces regression for learning continuous emotion dimensions

Nicolaou

Zafeiriou

Pantić

2013

Proceedings of the 21st ACM International Conference on Multimedia

Self Cite

View full text Add to dashboard Cite

Adopting continuous dimensional annotations for affective analysis has been gaining rising attention by researchers over the past years. Due to the idiosyncratic nature of this problem, many subproblems have been identified, spanning from the fusion of multiple continuous annotations to exploiting output-correlations amongst emotion dimensions. In this paper, we firstly empirically answer several important questions which have found partial or no answer at all so far in related literature. In more detail, we study the correlation of each emotion dimension (i) with respect to other emotion dimensions, (ii) to basic emotions (e.g., happiness, anger). As a measure for comparison, we use video and audio features. Interestingly enough, we find that (i) each emotion dimension is more correlated with other emotion dimensions rather than with face and audio features, and similarly (ii) that each basic emotion is more correlated with emotion dimensions than with audio and video features. Motivated by these findings, we present a novel regression algorithm (Correlated-Spaces Regression, CSR), inspired by Canonical Correlation Analysis (CCA) which learns outputcorrelations and performs supervised dimensionality reduction and multimodal fusion by (i) projecting features extracted from all modalities and labels onto a common space where their inter-correlation is maximised and (ii) learning mappings from the projected feature space onto the projected, uncorrelated label space.

show abstract

“…We consider face and head movements tracked by the state-of-the-arttrackers (i.e. the face tracker described in [25] and the head pose estimator described in [17]) and report on binary classification of video sequences into mimicry and non-mimicry categories based on the following widely-used methodology: two similarity-based methods (cross correlation as used in [22] and Generalised Time Warping [40]), and the state-of-the-art temporal classifier, Long Short Term Memory Recurrent Neural Network (LSTM-RNN) [32]. Performance of the methods is evaluated against the ground truth, representing human annotations of motor mimicry behaviour.…”

Section: A C C E P T E D Mmentioning

confidence: 99%