Pleasure-arousal-dominance driven facial expression simulation

Abstract

Expressing and recognizing affective states with respect to facial expressions is an important aspect in perceiving virtual humans as more natural and believable. Based on the results of an empirical study a system for simulating emotional facial expressions for a virtual human has been evolved. This system consists of two parts: (1) a control architecture for simulating emotional facial expressions with respect to Pleasure, Arousal, and Dominance (PAD) values, (2) an expressive output component for animating the virtual human's facial muscle actions called Action Units (AUs), modeled following the Facial Action Coding System (FACS). A large face repertoire of about 6000 faces arranged in PAD-space with respect to two dominance values (dominant vs. submissive) is obtained as a result of the empirical study. Using the face repertoire an approach towards realizing facial mimicry for a virtual human based on backward mapping AUs displaying an emotional facial expression on PAD-values is outlined. A preliminary evaluation of this first approach is realized with AUs corresponding to the basic emotions Happy and Angry.

Head and facial gestures synthesis using PAD model for an expressive talking avatar

“…Since the PAD model is originally designed for human emotion evaluation, it is the most popular and recommend way to obtain PAD values through human ratings [4,5,31]. Previous engineering approaches have gain convincible PAD values from human annotation [3] and predefined rules [26,38].…”

“…Despite the limited variability covered in PAD space, we propose to build a general mapping model between PAD descriptors and motion features, which can be easily extended to accommodate higher variability in the PAD continuum. Some preliminary study has shown that it is Visual speech with both prosodic head motion and facial expression ( a the random head motion is also synthesized by the sinusoidal function for each prosodic word, but with random generated amplitude and average position within the statistical range in Table 3, and random "peak" position within the prosodic word) ( b the facial expression in Exp3 is synthesized by the ANN-based predictor) possible to predict the PAD values automatically from different media, such as the backward mapping facial displaying emotions to PAD values [3], and the PAD-PEP-FAP framework can also be reversed to predict the PAD values from facial animations.…”

“…The three dimensions include pleasuredispleasure (P), arousal-nonarousal (A) and dominance-submissiveness (D). The PAD model has been applied in affective sate modeling [17], expressive speech synthesis [35], facial expression simulation [3] and expressive head and body movement generation [26]. In our previous work on expressive text-to-speech synthesis [38], a set of PAD heuristics was proposed for text semantics annotation, which will be reused in this paper.…”

“…the head and facial animation parameters). Different with previous studies that explore the use of PAD model in discrete regions with only two dimensions [3,26], we aims to build a general framework that is capable in covering all the PAD dimensions, explore the correlation between PAD dimensions and motion features in a quantitative way, and further apply the mapping of PAD to motion features in synthesizing the head and facial gestures for an expressive talking avatar.…”

Empathy and Its Modulation in a Virtual Human

“…Third, the Expression of Empathy by which the virtual human's multimodal behavior is triggered through the modulated empathic emotion. The presented empathy model is applied and evaluated in the context of a conversational agent scenario involving the virtual humans MAX [12] and EMMA [6] and a human interaction partner. Within this scenario, our model is realized for EMMA and allows her to empathize with MAX's emotions during his interaction with the human partner.…”