Selective Transfer Machine for Personalized Facial Action Unit Detection

Abstract

Automatic facial action unit (AFA) detection from video is a long-standing problem in facial expression analysis. Most approaches emphasize choices of features and classifiers. They neglect individual differences in target persons. People vary markedly in facial morphology (e.g., heavy versus delicate brows, smooth versus deeply etched wrinkles) and behavior. Individual differences can dramatically influence how well generic classifiers generalize to previously unseen persons. While a possible solution would be to train person-specific classifiers, that often is neither feasible nor theoretically compelling. The alternative that we propose is to personalize a generic classifier in an unsupervised manner (no additional labels for the test subjects are required). We introduce a transductive learning method, which we refer to Selective Transfer Machine (STM), to personalize a generic classifier by attenuating person-specific biases. STM achieves this effect by simultaneously learning a classifier and re-weighting the training samples that are most relevant to the test subject. To evaluate the effectiveness of STM, we compared STM to generic classifiers and to cross-domain learning methods in three major databases: CK+ [20], GEMEP-FERA [32] and RU-FACS [2]. STM outperformed generic classifiers in all.

Personalized Modeling of Facial Action Unit Intensity

“…In our future work, we plan to extend this approach so that it can perform feature decomposition from previously unseen subjects. We also plan to further assess the performance of our approach by comparing it to personalized models based on transfer learning (e.g., [16]) as well as investigate its generalization to high-dimensional facial features (i.e., appearance-based features).…”

“…Similarly, [13] proposed two transfer learning algorithms: inductive and transductive transfer learning for detection of pain and AUs, in both a semi-supervised and unsupervised settings. [16] proposed a personalized model for AU detection that is based on the Kernel Mean Matching technique. In this approach, an iterative minimization procedure is proposed to adapt the hyperplanes of generic SVM, trained using the labeled source data, to the target person.…”

Discriminant Incoherent Component Analysis

“…Action unit detection results in terms of F 1 score, as produced by each method, are reported in Table VI for each action unit along with the average performance over all AU classes. For comparison purposes, we choose to also include in Table VI the results reported in [65] for the same evaluation protocol for Selective Transfer Machine (STM), which is a recently published successful method for AU detection. The DICA achieves similar performance to that of STM 4 , while it outperforms all other methods.…”

Machine Learning Methods for Social Signal Processing

“…These static approaches are deemed context-insensitive as they focus on answering only one context question, i.e., how. Recently, (Chu et al 2013) proposed a transductive learning method, named Selective Transfer Machine (STM), where a SVM classifier for AU detection is personalized by attenuating person-specific biases, thus, simultaneously answering the context questions who and how. This is accomplished by learning the classifier and reweighting the training samples that are most relevant to the test subject during inference.…”