Effects of damping head movement and facial expression in dyadic conversation using real–time facial expression tracking and synthesized avatars

Abstract

When people speak with one another, they tend to adapt their head movements and facial expressions in response to each others' head movements and facial expressions. We present an experiment in which confederates' head movements and facial expressions were motion tracked during videoconference conversations, an avatar face was reconstructed in real time, and naive participants spoke with the avatar face. No naive participant guessed that the computer generated face was not video. Confederates' facial expressions, vocal inflections and head movements were attenuated at 1 min intervals in a fully crossed experimental design. Attenuated head movements led to increased head nods and lateral head turns, and attenuated facial expressions led to increased head nodding in both naive participants and confederates. Together, these results are consistent with a hypothesis that the dynamics of head movements in dyadicconversation include a shared equilibrium. Although both conversational partners were blind to the manipulation, when apparent head movement of one conversant was attenuated, both partners responded by increasing the velocity of their head movements.

Dyadic drumming across the lifespan reveals a zone of proximal development in children.

“…This result supports the claim that a reliable amount of variance in IAS accuracy is an emergent property of the interaction dynamics, which are influenced by the age composition of the dyads in ways yet unknown. In interpersonal interactions, individuals continuously need to adjust their own actions to the actions of others, which in turn are also influenced by the interaction partner (e.g., Boker et al, 2009;Nowak et al, 2005;Tognoli, Lagarde, de Guzman, & Kelso, 2007). In the context of the present task, coupled oscillators may provide a viable model for delineating age-based differences in interaction dynamics (e.g., Ashenfelter, Boker, Waddell, & Vitanov, 2009;Boker & Laurenceau, 2007).…”

“…In-phase coordination is generally more stable than anti-phase coordination, both in individual and dyadic contexts, and performance stability in both coordination modes has been found to decline as tapping frequency increases (e.g., Kelso, 1984;Schmidt et al, 1990;Schmidt, Christianson, Carello, & Baron, 1994;Schmidt & Turvey, 1994). The purpose of the present study was to investigate natural adjustment processes that occur between individuals when synchronizing with each other to reach a shared goal, while at the same time reducing the complexity of synchronization processes as they naturally occur in everyday interactions (e.g., Boker, Cohn, Theobald, Matthews, Brick, & Spies, 2009;Boker & Rotondo, 2003). Therefore, in the present study, individuals were asked to synchronize symmetrically (i.e., in-phase) at a tempo with which they felt comfortable.…”

A facial affect mapping engine

“…A similar facial puppetry system has been used for a psychological experiment that explored the effect on attenuation of head-pose and facial expression in photo-realistic videoconferencing scenarios [3]. Our system enables researches to conduct similar experiments, with natural and realistic blending of the modified face onto the original video background.…”

Computation of emotions in man and machines

“…Cohn (Boker et al 2009) presents an experiment that shows how people adapt their head movements and facial expressions in response to their partners in a social exchange to maintain equilibrium throughout the interaction. The experiment uses real-time automatic face tracking to drive a synthetic facial avatar in real time for communication in one direction and straightforward video conferencing for the return channel.…”

“…Cohn (Boker et al 2009) describes a novel experimental paradigm that allows for the systematic testing of dyadic interactions, an area that was previously hard to undertake because of the range of parameters involved. Frith (2009) shows how the brain's responses to robots are fundamentally different from those to humans and challenges us to develop robots that can produce an affective imprint in humans.…”