Context congruency and robotic facial expressions: Do effects on human perceptions vary across culture?

Abstract

We performed an experimental study (n=48) of the effects of context congruency on human perceptions of robotic facial expressions across cultures (Western and East Asian individuals). We found that context congruency had a significant effect on human perceptions, and that this effect varied by the emotional valence of the context and facial expression. Moreover, these effects occurred regardless of the cultural background of the participants. In short, there were predictable patterns in the effects of congruent/incongruent environmental context on perceptions of robot affect across Western and East Asian individuals. We argue that these findings fit with a dynamical systems view of social cognition as an emergent phenomenon. Taking advantage of such context effects may ease the constraints for developing culturallyspecific affective cues in human-robot interaction, opening the possibility to create culture-neutral models of robots and affective interaction.

Multimodal Integration of Emotional Signals from Voice, Body, and Context: Effects of (In)Congruence on Emotion Recognition and Attitudes Towards Robots

“…With the exception of a handful of studies (e.g., [6,46,47]) that examine the perception of robotic facial expressions in the presence of incongruent contextual information (i.e., movie clips or pictures), the perception of multimodal emotional signals from robots has mainly focused on studies comparing responses to unimodal versus congruent bimodal emotional stimuli (e.g., [23][24][25]). These studies have examined the perception of emotional expressions involving faces and voices [23], faces and gestures [24], gestures and voices [25] and face-voice-gesture [23] combinations.…”

“…A small number of HRI studies (e.g., [6,46,47]) have examined the perception of robotic facial expressions in the presence of incongruent contextual information. The context was manipulated by having participants watch emotioneliciting pictures or movie clips or listen to news clips with positive or negative emotional valence.…”

“…Overall, results showed that the recognition of robotic facial expressions is significantly better in the presence of congruent context, as opposed to no context or incongruent context. Providing incongruent context can even be worse than providing no context at all [47]. Furthermore, [46] showed that when the expressions of a robot are not appropriate given the context, subjects' judgments are more biased by the context than the expressions themselves.…”

“…Emotion elicitation using movie clips is a common experimental manipulation used in psychology studies of emotions [48], and has been successfully used in HRI studies to elicit emotional responses in healthy individuals in the laboratory (e.g. [47,49]). We also manipulated the emotional congruence of the multimodal reactions of the robot (consisting of vocal expressions and body postures) to each movie clip as follows (see Table 1): -In the congruent condition, the emotional valence of the multimodal reaction of the robot was congruent with the valence of the socio-emotional context of the interaction (elicited by the movie clip).…”

“…1). A number of previous studies have also suggested that the effects of congruence may vary by valence (e.g., [46,47,51]). We chose to investigate the emotions of happiness, sadness, and surprise for a number of reasons.…”