Gaze-Vergence-Controlled See-Through Vision in Augmented Reality

Wang, Zhimin; Zhao, Yuxin; Lu, Feng

doi:10.1109/tvcg.2022.3203110

Cited by 15 publications

(6 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Previous works have shown the feasibility of visual depth estimations in the VR headsets by using binocular disparity and stereoscopic vergence in VR headset [10,40]. The concept of visual depth as a new interaction input has explored in the previous works, either by defining a semi-transparent window at a different focal depth [26,41,42], tracking voluntary vergence movements [3,13], or matching the vergence changes with the depth changes of a moving objects [34] in VR. However, all of these methods lack an end-to-end UI design to guide the users how to actively manipulate their visual depth.…”

Section: Gaze-based Vr Interactionmentioning

confidence: 99%

“…More recent works on gaze-based VR/AR interaction demonstrate the great potential of visual depth as an interaction input to solve the Midas touch problem. These methods either guide the user to look at physical or virtual objects at different depths [3,26,34,41,42] or rely on voluntary eye convergence and divergence [13,15] by asking the users focusing on the nose or imagining to fixate on some point behind the display plane. However, these works lack an intuitive and systematic User Interface (UI) design to guide the users in manipulating their visual depth, leading to limited application scenarios and potentially user frustration.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

FocusFlow: 3D Gaze-Depth Interaction in Virtual Reality Leveraging Active Visual Depth Manipulation

Zhang,

Chen,

Shaffer

et al. 2024

Proceedings of the CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

we propose two learning strategies that use different visual cues to help users master visual depth control. Our user studies on 24 participants demonstrate the usability of our proposed virtual window concept as a gaze-depth interaction method. In addition, our findings reveal that the user experience can be enhanced through an effective learning process with adaptive visual cues, helping users to develop muscle memory for this brand-new input mechanism. We conclude the paper by discussing strategies to optimize learning and potential research topics of gaze-depth interaction. CCS CONCEPTS• Human-centered computing → HCI design and evaluation methods; Virtual reality; Interaction design theory, concepts and paradigms.

show abstract

Section: Gaze-based Vr Interactionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

FocusFlow: 3D Gaze-Depth Interaction in Virtual Reality Leveraging Active Visual Depth Manipulation

Zhang,

Chen,

Shaffer

et al. 2024

Proceedings of the CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

show abstract

“…To ensure that the eyes in x f ro indeed look at (0, 0) with the head orientations unchanged, we introduce a reference face image x ref , which has eyeballs looking at (0, 0) and the same head orientation and identity as x, to constrain the generation of x f ro . We maximize Multi-Scale Structural Similarity (MS-SSIM) (Wang, Simoncelli, and Bovik 2003) between x ref and x f ro :…”

Section: Gazementioning

confidence: 99%

“…Gaze information is important for real applications. It indicates the direction or position at which a person is looking, and is widely used in many scenarios, such as augmented reality (Wang, Zhao, and Lu 2022) and autonomous driving (Mole et al 2021). To obtain this information, a number of gaze estimation methods have been proposed.…”

Section: Introductionmentioning

confidence: 99%

Gaze from Origin: Learning for Generalized Gaze Estimation by Embedding the Gaze Frontalization Process

Xu,

2024

AAAI

View full text Add to dashboard Cite

Gaze estimation aims to accurately estimate the direction or position at which a person is looking. With the development of deep learning techniques, a number of gaze estimation methods have been proposed and achieved state-of-the-art performance. However, these methods are limited to within-dataset settings, whose performance drops when tested on unseen datasets. We argue that this is caused by infinite and continuous gaze labels. To alleviate this problem, we propose using gaze frontalization as an auxiliary task to constrain gaze estimation. Based on this, we propose a novel gaze domain generalization framework named Gaze Frontalization-based Auxiliary Learning (GFAL) Framework which embeds the gaze frontalization process, i.e., guiding the feature so that the eyeball can rotate and look at the front (camera), without any target domain information during training. Experimental results show that our proposed framework is able to achieve state-of-the-art performance on gaze domain generalization task, which is competitive with or even superior to the SOTA gaze unsupervised domain adaptation methods.

show abstract

“…The human eye gaze tells what a person’s interests are, and it can be used as a medium for non-verbal communication. As a means of communication, the gaze can be used in many areas, such as human–computer interaction [ 1 ], human–robot interaction [ 2 ], virtual reality [ 3 ], augmented reality [ 4 ], and autonomous driving [ 5 ]. The goal of the gaze estimation task in computer vision is to estimate the gaze information of a subject from an input image that includes the subject’s face or eye(s).…”

Section: Introductionmentioning

confidence: 99%

ArbGaze: Gaze Estimation from Arbitrary-Sized Low-Resolution Images

Kim

Chang

2022

Sensors

View full text Add to dashboard Cite

The goal of gaze estimation is to estimate a gaze vector from an image containing a face or eye(s). Most existing studies use pre-defined fixed-resolution images to estimate the gaze vector. However, images captured from in-the-wild environments may have various resolutions, and variation in resolution can degrade gaze estimation performance. To address this problem, a gaze estimation method from arbitrary-sized low-resolution images is proposed. The basic idea of the proposed method is to combine knowledge distillation and feature adaptation. Knowledge distillation helps the gaze estimator for arbitrary-sized images generate a feature map similar to that from a high-resolution image. Feature adaptation makes creating a feature map adaptive to various resolutions of an input image possible by using a low-resolution image and its scale information together. It is shown that combining these two ideas improves gaze estimation performance substantially in the ablation study. It is also demonstrated that the proposed method can be generalized to other popularly used gaze estimation models through experiments using various backbones.

show abstract

Gaze-Vergence-Controlled See-Through Vision in Augmented Reality

Cited by 15 publications

References 61 publications

FocusFlow: 3D Gaze-Depth Interaction in Virtual Reality Leveraging Active Visual Depth Manipulation

FocusFlow: 3D Gaze-Depth Interaction in Virtual Reality Leveraging Active Visual Depth Manipulation

Gaze from Origin: Learning for Generalized Gaze Estimation by Embedding the Gaze Frontalization Process

ArbGaze: Gaze Estimation from Arbitrary-Sized Low-Resolution Images

Contact Info

Product

Resources

About