Measuring motion-to-photon latency for sensorimotor experiments with virtual reality systems

Abstract

Consumer virtual reality (VR) systems are increasingly being deployed in research to study sensorimotor behaviours, but properties of such systems require verification before being used as scientific tools. The 'motion-to-photon' latency (the lag between a user making a movement and the movement being displayed within the display) is a particularly important metric as temporal delays can degrade sensorimotor performance. Extant approaches to quantifying this measure have involved the use of bespoke software and hardware and produce a single measure of latency and ignore the effect of the motion prediction algorithms used in modern VR systems. This reduces confidence in the generalisability of the results. We developed a novel, system-independent, high-speed camera-based latency measurement technique to co-register real and virtual controller movements, allowing assessment of how latencies change through a movement. We applied this technique to measure the motion-to-photon latency of controller movements in the HTC Vive, Oculus Rift, Oculus Rift S, and Valve Index, using the Unity game engine and SteamVR. For the start of a sudden movement, all measured headsets had mean latencies between 21-42ms. Once motion prediction could account for the inherent delays, the latency was functionally reduced to 2-13ms, and our technique revealed this reduction occurs within ~25-58ms of movement onset. Our findings indicate that sudden accelerations (e.g. movement onset, impacts and direction changes) will increase latencies and lower spatial accuracy. Our technique allows researchers to measure these factors and determine the impact on their experimental design before collecting sensorimotor data from VR systems.

Design of Virtual Reality-Based Hippotherapy Simulator Exergaming Software and Its Controller for Rehabilitation of Children with Cerebral Palsy in Indonesia: An Engineering Concept

“…We did the experiment 5 times and averaged the results. We get the average latency results of about 85 ms, which is a quite reasonable motion-to-photon latency compared to other device such as HMD which has about 25-50 ms [37].…”

“…2 We consider the use of VR HMD to have a high immersivity, while a simple monitor is not immersive. 3 The motion-to-photon (MTP) delay of Nintendo Wii and most of the VR Headset is under 50 ms. [37] 4 We consider the use of VR HMD to be a medium risks when combined with safety harness because they obstruct the view of the participants. 5 They only ask the participants to correct the posture by the therapist to ensure safety.…”

Visual and haptic cues in processing occlusion

“…The VR system was implemented using Unity 2017.4.15f1. The latency was within 25 ms after the movement onset and then within 5 ms on average (Warburton et al, 2022). The participant sat on a chair and was confronted with a wooden board on a desk 45 cm in front of them.…”

Transfer Functions and Event Fields in XR: The PHANTOMATRIX Framework for Quantifying Perception Convergence

“…This is mainly caused by the latency 𝐷 𝑇𝑙𝑎𝑡 between the motion of the observer and the appearance of the change in the display (motion to photon). But this is not a single limitation, this includes many steps like sensor sampling 𝐷 𝑇𝑠𝑒𝑛𝑠 , data transfer 𝐷 𝑇𝑡𝑟𝑎𝑛𝑠 , data processing 𝐷 𝑇𝑑𝑖𝑠𝑝 , changing and rendering the virtual environment 𝐷 𝑇𝑟𝑒𝑛𝑑 and finally the display of the rendered images 𝐷 𝑇𝑑𝑖𝑠𝑝 [22]. Every single step will induce a latency and the motion to photon latency is the sum of all: 𝐷 𝑇𝑙𝑎𝑡 = 𝐷 𝑇𝑠𝑒𝑛𝑠 + 𝐷 𝑇𝑡𝑟𝑎𝑛𝑠 + 𝐷 𝑇𝑝𝑟𝑜𝑐 + 𝐷 𝑇𝑟𝑒𝑛𝑑 + 𝐷 𝑇𝑑𝑖𝑠𝑝 .…”