No abstract

Spontaneous facial expressions differ from posed expressions in both which muscles are moved, and in the dynamics of the movement. Advances in the field of automatic facial expression measurement will require development and assessment on spontaneous behavior. Here we present preliminary results on a task of facial action detection in spontaneous facial expressions. We employ a user independent fully automatic system for real time recognition of facial actions from the Facial Action Coding System (FACS). The system automatically detects frontal faces in the video stream and coded each frame with respect to 20 Action units. The approach applies machine learning methods such as support vector machines and AdaBoost, to texture-based image representations. The output margin for the learned classifiers predicts action unit intensity. Frame-by-frame intensity measurements will enable investigations into facial expression dynamics which were previously intractable by human coding.

No abstract

We present a systematic comparison of machine learning methods applied to the problem of fully automatic recognition of facial expressions. We explored recognition of facial actions from the Facial Action Coding System (FACS), as well as recognition of full facial expressions. Each videoframe is first scanned in real-time to detect approximately upright-frontal faces. The faces found are scaled into image patches of equal size, convolved with a bank of Gabor energy filters, and then passed to a recognition engine that codes facial expressions into 7 dimensions in real time: neutral, anger, disgust, fear, joy, sadness, surprise. We report results on a series of experiments comparing recognition engines, including AdaBoost, support vector machines, linear discriminant analysis, as well as feature selection techniques. Best results were obtained by selecting a subset of Gabor filters using AdaBoost and then training Support Vector Machines on the outputs of the filters selected by AdaBoost. The generalization performance to new subjects for recognition of full facial expressions in a 7-way forced choice was 93% correct, the best performance reported so far on the DFAT-504 dataset. We also applied the system to fully automated facial action coding. The present system classifies 18 action units, whether they occur singly or in combination with other actions. The system obtained a mean agreement rate of 94.5% on a FACS-coded dataset of posed expressions (DFAT-504). The outputs of the classifiers change smoothly as a function of time and thus can be used to measure facial expression dynamics.

The pathophysiology of Parkinson’s disease (PD) is known to involve altered patterns of neuronal firing and synchronization in cortical-basal ganglia circuits. One window into the nature of the aberrant temporal dynamics in the cerebral cortex of PD patients can come from analysis of the patients electroencephalography (EEG). Rather than using spectral-based methods, we used data models based on delay differential equations (DDE) as non-linear time-domain classification tools to analyze EEG recordings from PD patients on and off dopaminergic therapy and healthy individuals. Two sets of 50 1-s segments of 64-channel EEG activity were recorded from nine PD patients on and off medication and nine age-matched controls. The 64 EEG channels were grouped into 10 clusters covering frontal, central, parietal, and occipital brain regions for analysis. DDE models were fitted to individual trials, and model coefficients and error were used as features for classification. The best models were selected using repeated random sub-sampling validation and classification performance was measured using the area under the ROC curve A′. In a companion paper, we show that DDEs can uncover hidden dynamical structure from short segments of simulated time series of known dynamical systems in high noise regimes. Using the same method for finding the best models, we found here that even short segments of EEG data in PD patients and controls contained dynamical structure, and moreover, that PD patients exhibited a greater dynamic range than controls. DDE model output on the means from one set of 50 trials provided nearly complete separation of PD patients off medication from controls: across brain regions, the area under the receiver-operating characteristic curves, A′, varied from 0.95 to 1.0. For distinguishing PD patients on vs. off medication, classification performance A′ ranged from 0.86 to 1.0 across brain regions. Moreover, the generalizability of the model to the second set of 50 trials was excellent, with A′ ranging from 0.81 to 0.94 across brain regions for controls vs. PD off medication, and from 0.62 to 0.82 for PD on medication vs. off. Finally, model features significantly predicted individual patients’ motor severity, as assessed with standard clinical rating scales.