Videoconference and Embodied VR: Communication Patterns Across Task and Medium

Abstract

Videoconference has become the dominant technology for remote meetings. Embodied Virtual Reality is a potential alternative that employs motion tracking in order to place people in a shared virtual environment as avatars. This paper describes a 210 participant study focused on behavioral measures that compares multiparty interaction in videoconference and embodied VR across a range of task types: a factual intellective task, a subjective judgment task and two negotiation tasks, one with visual grounding. It uses state-of-the-art body, face and finger tracking to drive the avatars in VR and a carefully matched videoconferencing implementation. Significant behavioral differences are observed. These include increased activity in videoconference related to maintaining the social connection: more person directed gaze and increased verbal and nonverbal backchannel behavior. Videoconference also had reduced conversational overlap, increased self-adaptor gestures and reduced deictic gestures as compared with embodied VR. Potential explanations and implications are discussed.

Videoconference Fatigue: A Conceptual Analysis

“…Problems with Nonverbal Cues: In VC, both the perception and production of non-verbal cues such as eye contact [ 14 , 89 ], vocal backchannel signals or facial expressions and gestures [ 89 , 90 , 91 ], indicating agreement and affection [ 92 ] can be hindered due to audio or video properties such as small windows on the screen. Increased efforts to produce and perceive those cues most likely lead to fatigue [ 14 , 15 , 46 , 93 ].…”

A state of the art and scoping review of embodied information behavior in shared, co-present extended reality experiences

“…[86] Gesture 13 26 [35], [40], [42], [45]- [48], [50], [51], [53], [69], [74], [84] Voice 9 18 [27], [39], [44], [51], [54], [60], [62], [79] The selected studies utilized 15 different types of system outputs. The five most common system outputs included: gaze visualization (64%, n=32), avatar representation (24%, n= 12), use of or as controller (16%, n=8), placement of annotations (10%, n=5), and display of facial expression (10%, n=5).…”

“…[37], [43], [45], [48], [50], [59], [62], [63], [76], [83], [84], [86] Controller 8 16 [34], [35], [39], [42], [46], [57], [70], [80] Place annotations 5 10…”

“…Eye Trackers Tobii Eye Tracker 11 22% [27], [35], [38], [43], [57], [61], [64], [72], [74], [75], [78] Pupil Labs eyetracker 7 14% [38], [41]- [43], [49], [71], [73] aGlass Eye Tracker 4 8% [44], [53], [55], [56] Advanced realtime tracking (ART) system 1 2% [77] Ergoneers Dikablis gaze tracker 1 2% [67] Eye Tracking Enabled HMDs HTC Vive Pro & Pro Eye 13 26% [25], [37], [41]- [43], [45], [46], [48], [54]- [58] Microsoft HoloLens 12 24% [26], [34]- [37], [39]- [44], [47] Magic Leap 3 6% [46]- [48] FOVE VR 2 4% [50], [70] BinaryVR HMD 1 2% [50] Other Approaches Gaze estimation from head position 6 12% [52], [59], [60], [62],…”

Video Conference System in Mixed Reality Using a Hololens

“…Furthermore, a more immersive video conference system based on telepresence technology increases the natural perception of the environment through communication media. For telepresence technology [9], some mature commercial products have emerged, such as Polycom RealPresence Immersive Studio [10], Cisco Immersive TelePresence [11]. They generally need to build a conference room with relatively large space, adopt an integrated conference table, hidden microphone, and speaker, so that participants do not need to deliberately close to the microphone, maintain a natural way of communication, and provide excellent listening and position recognition function to realize a sense of space.…”