Leveling the Playing Field: A Comparative Reevaluation of Unmodified Eye Tracking as an Input and Interaction Modality for VR

Fernandes, Ajoy S.; Murdison, T. Scott; Proulx, Michael J.

doi:10.1109/tvcg.2023.3247058

Cited by 25 publications

(9 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the context of our low vision project, we were particularly interested by the ability of a subject to select a target with different kinds of techniques. Although the ability to select targets with different kinds of controllers has been heavily investigated in the VR literature with normally sighted persons ( Fernandes, Murdison, & Proulx, 2023 ; Yu et al, 2018 ), it seems that this topic has been overlooked with low vision persons.…”

Section: Resultsmentioning

confidence: 99%

PTVR – A software in Python to make virtual reality experiments easier to build and more reproducible

Castet,

Termoz-Masson,

Vizcay

et al. 2024

Journal of Vision

View full text Add to dashboard Cite

Researchers increasingly use virtual reality (VR) to perform behavioral experiments, especially in vision science. These experiments are usually programmed directly in so-called game engines that are extremely powerful. However, this process is tricky and time-consuming as it requires solid knowledge of game engines. Consequently, the anticipated prohibitive effort discourages many researchers who want to engage in VR. This paper introduces the Perception Toolbox for Virtual Reality (PTVR) library, allowing visual perception studies in VR to be created using high-level Python script programming. A crucial consequence of using a script is that an experiment can be described by a single, easy-to-read piece of code, thus improving VR studies' transparency, reproducibility, and reusability. We built our library upon a seminal open-source library released in 2018 that we have considerably developed since then. This paper aims to provide a comprehensive overview of the PTVR software for the first time. We introduce the main objects and features of PTVR and some general concepts related to the three-dimensional (3D) world. This new library should dramatically reduce the difficulty of programming experiments in VR and elicit a whole new set of visual perception studies with high ecological validity.

show abstract

Section: Resultsmentioning

confidence: 99%

PTVR – A software in Python to make virtual reality experiments easier to build and more reproducible

Castet,

Termoz-Masson,

Vizcay

et al. 2024

Journal of Vision

View full text Add to dashboard Cite

show abstract

“…Hence, it is a two-step procedure where the user’s gaze acts as a cursor and the selection is triggered by an additional modality. Various modalities have been investigated, including eye blinks [71], speech [63, 96, 122], head movements [115, 121], handheld devices [29, 64], tongue-based interfaces [34], body movements [57], and hand gestures such as the pinch gesture [64, 82, 99]. Additional gestures, such as gaze-hand alignment [74] confirm target selection either by aligning a ray that emanates from the user’s hand with their gaze (Gaze&Handray) or aligning the user’s finger with their gaze (Gaze&Finger) [117].…”

Section: Related Workmentioning

confidence: 99%

“…To mitigate the Midas touch, HCI researchers have proposed the use of an external trigger as a form of manual input [128]. This solution has been implemented in a variety of ways from voluntary blinks [71] to simply pressing a button on a physical controller [29] (see Section 2.1). These techniques improve selection accuracy, however, they require physical exertion hence potentially introducing fatigue over extended use [45].…”

Section: Introductionmentioning

confidence: 99%

Towards an Eye-Brain-Computer Interface: Combining Gaze with the Stimulus-Preceding Negativity for Target Selections in XR

Reddy,

Proulx,

Hirshfield

et al. 2024

Preprint

Self Cite

View full text Add to dashboard Cite

Gaze-assisted interaction techniques enable intuitive selections without requiring manual pointing but can result in unintended selections, known as Midas touch. A confirmation trigger eliminates this issue but requires additional physical and conscious user effort. Brain-computer interfaces (BCIs), particularly passive BCIs harnessing anticipatory potentials such as the Stimulus-Preceding Negativity (SPN) - evoked when users anticipate a forthcoming stimulus - present an effortless implicit solution for selection confirmation. Within a VR context, our research uniquely demonstrates that SPN has the potential to decode intent towards the visually focused target. We reinforce the scientific understanding of its mechanism by addressing a confounding factor - we demonstrate that the SPN is driven by the user’s intent to select the target, not by the stimulus feedback itself. Furthermore, we examine the effect of familiarly placed targets, finding that SPN may be evoked quicker as users acclimatize to target locations; a key insight for everyday BCIs.CCS CONCEPTSHuman-centered computing→Virtual reality;Mixed / augmented reality;Accessibility technologies;Interaction techniques.ACM Reference FormatG. S. Rajshekar Reddy, Michael J. Proulx, Leanne Hirshfield, and Anthony J. Ries. 2024. Towards an Eye-Brain-Computer Interface: Combining Gaze with the Stimulus-Preceding Negativity for Target Selections in XR. InProceedings of the CHI Conference on Human Factors in Computing Systems (CHI ‘24), May 11–16, 2024, Honolulu, HI, USA. ACM, New York, NY, USA, 17 pages.https://doi.org/10.1145/3613904.3641925

show abstract

“…issue 2, June 2024 Furthermore, the Meta Quest 2, a leading VR device, has garnered attention for further research and development. Efforts have been made to measure controller precision [26], hand movement accuracy [27], eye-tracking evaluations [28], integration of haptic features [29], [30], and the creation of 3D Avatar applications [31].…”

Section: Introductionmentioning

confidence: 99%

Hand Sign Interpretation through Virtual Reality Data Processing

Tju,

Shalih

2024

Jurnal Ilmu Komputer dan Informasi

View full text Add to dashboard Cite

The research lays the groundwork for further advancements in VR technology, aiming to develop devices capable of interpreting sign language into speech via intelligent systems. The uniqueness of this study lies in utilizing the Meta Quest 2 VR device to gather primary hand sign data, subsequently classified using Machine Learning techniques to evaluate the device's proficiency in interpreting hand signs. The initial stages emphasized collecting hand sign data from VR devices and processing the data to comprehend sign patterns and characteristics effectively. 1021 data points, comprising ten distinct hand sign gestures, were collected using a simple application developed with Unity Editor. Each data contained 14 parameters from both hands, ensuring alignment with the headset to prevent hand movements from affecting body rotation and accurately reflecting the user's facing direction. The data processing involved padding techniques to standardize varied data lengths resulting from diverse recording periods. The Interpretation Algorithm Development involved Recurrent Neural Networks tailored to data characteristics. Evaluation metrics encompassed Accuracy, Validation Accuracy, Loss, Validation Loss, and Confusion Matrix. Over 15 epochs, validation accuracy notably stabilized at 0.9951, showcasing consistent performance on unseen data. The implications of this research serve as a foundation for further studies in the development of VR devices or other wearable gadgets that can function as sign language interpreters.

show abstract

Leveling the Playing Field: A Comparative Reevaluation of Unmodified Eye Tracking as an Input and Interaction Modality for VR

Cited by 25 publications

References 61 publications

PTVR – A software in Python to make virtual reality experiments easier to build and more reproducible

PTVR – A software in Python to make virtual reality experiments easier to build and more reproducible

Towards an Eye-Brain-Computer Interface: Combining Gaze with the Stimulus-Preceding Negativity for Target Selections in XR

Hand Sign Interpretation through Virtual Reality Data Processing

Contact Info

Product

Resources

About