Contextual Recognition of Robot Emotions

Abstract

In the field of human-robot interaction, socially interactive robots are often equipped with the ability to detect the affective states of users, the ability to express emotions through the use of synthetic facial expressions, speech and textual content, and the ability for imitating and social learning. Past work on creating robots that can make convincing emotional expressions has concentrated on the quality of those expressions, and on assessing people's ability to recognize them. Previous recognition studies presented the facial expressions of the robots in neutral contexts, without any strong emotional valence (e.g., emotionally valenced music or video). It is therefore worth empirically exploring whether observers' judgments of the facial cues of a robot would be affected by a surrounding emotional context. This thesis takes its inspiration from the contextual effects found on the interpretation of the expressions on human faces and computer avatars, and looks at the extent to which they also apply to the interpretation of the facial expressions of a mechanical robot head. The kinds of contexts that affect the recognition of robot emotional expressions, the circumstances under which such contextual effects occur, and the relationship between emotions and the surrounding situation, are observed and analyzed in a series of 11 experiments. In these experiments, the FACS (Facial Action Coding System) (Ekman and Friesen, 2002) was applied to set up the parameters of the servos to make the robot head produce sequences of facial expressions. Four different emotional surrounding or preceding contexts were used (i.e., recorded BBC News pieces, selected affective pictures, classical music pieces and film clips). This thesis provides evidence that observers' judgments about the facial expressions of a robot can be affected by a surrounding emotional context. From a psychological perspective, the contextual effects found on the robotic facial expressions based on the FACS, indirectly support the claims that human emotions are both 4 biologically based and socially constructed. From a robotics perspective, it is argued that the results obtained from the analyses will be useful for guiding researchers to enhance the expressive skills of emotional robots in a surrounding emotional context. This thesis also analyzes the possible factors contributing to the contextual effects found in the original 11 experiments.Some future work, including four new experiments (a preliminary experiment designed to identify appropriate contextual materials and three further experiments in which factors likely to affect a context effect are controlled one by one) is also proposed in this thesis.

The effects of culture and context on perceptions of robotic facial expressions

“…In particular, Feara notoriously difficult emotion to convey via robotic facial expressionsincreased to nearly 100% with added context, regardless of cultural background of the subjects. These findings concur with previously reported context effects in both humans/avatars (Righart & de Gelder, 2008;Barrett et al, 2011;Lee et al 2012) as well as robots (Zhang & Sharkey, 2011). We were also able to replicate these effects in a companion study in which we looked at the effects of both incongruent and congruent context on people's perceptions of a robots affective facial expressions which showed significant differences across all emotions, except for surprise .…”

“…Anger, Surprise) with high accuracy. Elsewhere, Zhang and Sharkey (2011) have evaluated the effects of context on robotic facial expression identification by humans, and Embgen et al (2012) have conducted robotic studies on emotional body language in lieu of facial expressions.…”

Context congruency and robotic facial expressions: Do effects on human perceptions vary across culture?

“…This trend varied by the valence of the expression [33]. Recent work has shown the importance of context in perceptions of robotic facial expressions across cultures as well [11,37].…”

Design of a Huggable Social Robot with Affective Expressions Using Projected Images

“…The videos presented the robot displaying one type of visual feedback without being contextualized by the environment or interaction. A previous study showed evidence that the context could affect the participant's recognition of the robot's expressions [63]. For this reason, we avoided influencing the participant's choices by adding elements related to interaction, such as by showing the happy state after hugging or the sad one after hitting.…”