Eye See What You See: Exploring How Bi-Directional Augmented Reality Gaze Visualisation Influences Co-Located Symmetric Collaboration

Jing, Allison; May, Kieran; Lee, Gun; Billinghurst, Mark

doi:10.3389/frvir.2021.697367

Cited by 32 publications

(5 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The participants could interact seamlessly with remote partners through pseudo‐eye contact. Similar to previous studies, involving codriver (Maurer et al, 2014) and visual searching and matching tasks (Jing et al, 2021), the current tasks requiring coordinating spatial referencing and attention benefit from this shared gaze.…”

Section: Discussionsupporting

confidence: 74%

Pseudo‐mutual gazing enhances interbrain synchrony during remote joint attention tasking

Chuang

Hsu

2023

Brain and Behavior

View full text Add to dashboard Cite

IntroductionMutual gaze enables people to share attention and increase engagement during social interactions through intentional and implicit messages. Although previous studies have explored gaze behaviors and neural mechanisms underlying in‐person eye contact, the growing prevalence of remote communication has raised questions about how to establish mutual gaze remotely and how the brains of interacting individuals synchronize.MethodsTo address these questions, we conducted a study using eye trackers to create a pseudo‐mutual gaze channel that mirrors the gazes of each interacting dyad on their respective remote screens. To demonstrate fluctuations in coupling across brains, we incorporated electroencephalographic hyperscanning techniques to simultaneously record the brain activity of interacting dyads engaged in a joint attention task in player‐observer, collaborative, and competitive modes.ResultsOur results indicated that mutual gaze could improve the efficiency of joint attention activities among remote partners. Moreover, by employing the phase locking value, we could estimate interbrain synchrony (IBS) and observe low‐frequency couplings in the frontal and temporal regions that varied based on the interaction mode. While dyadic gender composition significantly affected gaze patterns, it did not impact the IBS.ConclusionThese results provide insight into the neurological mechanisms underlying remote interaction through the pseudo‐mutual gaze channel and have significant implications for developing effective online communication environments.

show abstract

Section: Discussionsupporting

confidence: 74%

Pseudo‐mutual gazing enhances interbrain synchrony during remote joint attention tasking

Chuang

Hsu

2023

Brain and Behavior

View full text Add to dashboard Cite

show abstract

“…Therefore, measuring cognitive load or work load in HCI research is popular. For example, Jing et al (2021), adopted SMEQ questionnaire (Sauro and Dumas, 2009) to measure workload in evaluating three bidirectional collaborative gaze visualizations with three levels of gaze behaviors for co-located collaboration. The cognitive load in their research task is extraneous cognitive load as the researchers manipulated the manner of communicating cues.…”

Section: Cognitive Load Theorymentioning

confidence: 99%

The impact of virtual agents’ multimodal communication on brain activity and cognitive load in Virtual Reality

Chang¹,

Bai²,

Zhang³

et al. 2022

Front. Virtual Real.

Self Cite

View full text Add to dashboard Cite

Related research has shown that collaborating with Intelligent Virtual Agents (IVAs) embodied in Augmented Reality (AR) or Virtual Reality (VR) can improve task performance and reduce task load. Human cognition and behaviors are controlled by brain activities, which can be captured and reflected by Electroencephalogram (EEG) signals. However, little research has been done to understand users’ cognition and behaviors using EEG while interacting with IVAs embodied in AR and VR environments. In this paper, we investigate the impact of the virtual agent’s multimodal communication in VR on users’ EEG signals as measured by alpha band power. We develop a desert survival game where the participants make decisions collaboratively with the virtual agent in VR. We evaluate three different communication methods based on a within-subject pilot study: 1) a Voice-only Agent, 2) an Embodied Agent with speech and gaze, and 3) a Gestural Agent with a gesture pointing at the object while talking about it. No significant difference was found in the EEG alpha band power. However, the alpha band ERD/ERS calculated around the moment when the virtual agent started speaking indicated providing a virtual body for the sudden speech could avoid the abrupt attentional demand when the agent started speaking. Moreover, a sudden gesture coupled with the speech induced more attentional demands, even though the speech was matched with the virtual body. This work is the first to explore the impact of IVAs’ interaction methods in VR on users’ brain activity, and our findings contribute to the IVAs interaction design.

show abstract

“…In another study, Jing et al compared different types of gaze visualizations and the effect of gaze behaviours in a co-located environment with AR head-mounted displays and showed that gaze markers are helpful indicators for intentions and joint attention [31]. Erickson et al analyzed the effectiveness of gaze rays on target identification with a simulated gaze and examined the error types [32].…”

Section: Communication and Collaborationmentioning

confidence: 99%

Intraoperative gaze guidance with mixed reality

Acar,

Atoum,

Reed

et al. 2023

Healthcare Tech Letters

View full text Add to dashboard Cite

Efficient communication and collaboration are essential in the operating room for successful and safe surgery. While many technologies are improving various aspects of surgery, communication between attending surgeons, residents, and surgical teams is still limited to verbal interactions that are prone to misunderstandings. Novel modes of communication can increase speed and accuracy, and transform operating rooms. A mixed reality (MR) based gaze sharing application on Microsoft HoloLens 2 headset that can help expert surgeons indicate specific regions, communicate with decreased verbal effort, and guide novices throughout an operation is presented. The utility of the application is tested with a user study of endoscopic kidney stone localization completed by urology experts and novice surgeons. Improvement is observed in the NASA task load index surveys (up to 25.23%), in the success rate of the task (6.98% increase in localized stone percentage), and in gaze analyses (up to 31.99%). The proposed application shows promise in both operating room applications and surgical training tasks.

show abstract

Eye See What You See: Exploring How Bi-Directional Augmented Reality Gaze Visualisation Influences Co-Located Symmetric Collaboration

Cited by 32 publications

References 35 publications

Pseudo‐mutual gazing enhances interbrain synchrony during remote joint attention tasking

Pseudo‐mutual gazing enhances interbrain synchrony during remote joint attention tasking

The impact of virtual agents’ multimodal communication on brain activity and cognitive load in Virtual Reality

Intraoperative gaze guidance with mixed reality

Contact Info

Product

Resources

About