Injecting Nonverbal Mimicry with Hybrid Avatar-Agent Technologies

Technologies for Social Augmentations in User-Embodied Virtual Reality

“…To our knowledge, the combined replication of body movement, facial expression, and gaze behavior in a shared and HMD-based avatar-mediated system has not yet been presented, and represents a first requirement (RE1) in order to support variable social augmentations features. With regard to the augmentation of behaviors, previous approaches are limited to linear transformations, such as dampening or amplifying of facial cues [Boker et al 2009;Oh et al 2016;Roth et al 2018c] or modification based on intra-personal information [Bailenson et al 2006]. Bailenson et al [2008] discussed the presentation of real-time information of social cues during communication, and Roth et al [2018b] implemented a speaker-listener based paradigm of gaze augmentation.…”

“…The replication of behavior was limited to keyboard input, and participants reported a lack of feedback due to missing body movement and body language [Tromp et al 1998]. Steptoe, Roberts and colleagues investigated additional modalities for social interaction by including eye gaze [Hart et al 2018a], © 2018 Hart, et al) c) augmented nonverbal mimicry [Roth et al 2018c] (© 2018 Roth, et al), d) visual transformation, substitution, and amplification of social behaviors [Roth et al 2018a] (© 2018 IEEE /Roth, et al). et al 2009].…”

“…HAAT thus takes into account human and technological aspects to actively mediate virtual communication in the social dimension. Roth et al further explored hybrid and augmented gaze [Roth et al 2018b], augmented mimicry [Roth et al 2018c] (Fig. 2 c), as well as visual transformations of social phenomena using HAAT in multiuser VR that lead to increased social presence [Roth et al 2018a] (Fig.…”

“…Nonverbal mimicry can be described as the motor imitation of another's behavior and is associated with positive impacts such as affinity, liking, and rapport [Lakin et al 2003]. To induce artificial mimicry, we based the implementation on the approach from Roth et al [2018c]. While [Roth et al 2018c] used a naïve periodic induction of mimicry, the present prototype stresses the decision-making process of the underlying SIMC component, which makes decisions on the basis of the adequacy of a manipulation.…”

Exploring the Use of a Robust Depth-sensor-based Avatar Control System and its Effects on Communication Behaviors

“…The non-verbal communication behavior is usually presented in the mutual conversion through face-to-face, video conferencing [Whittaker 2003] or embodied avatar in VR [Dodds et al 2011;Fabri et al 2002;Roth et al 2018;Smith and Neff 2018]. The non-verbal cues delivered by the virtual characters in the collaborative virtual environment influence the efficiency of task performance [Roth et al 2018;Smith and Neff 2018]. The mirror is usually used in the singleuser scenario [Collingwoode-Williams et al 2017;Kilteni et al 2013;Maister et al 2015] to evaluate the communication behavior such as non-verbal cue.…”