Visuomotor Interactions and Perceptual Judgments in Virtual Reality Simulating Different Levels of Gravity

Abstract

Virtual reality is used to manipulate sensorimotor interactions in a controlled manner. A critical issue is represented by the extent to which virtual scenarios must conform to physical realism to allow ecological human-machine interactions. Among the physical constraints, Earth gravity is one of the most pervasive and significant for sensorimotor coordination. However, it is still unclear whether visual perception is sensitive to the level of gravity acting on target motion displayed in virtual reality, given the poor visual discrimination of accelerations. To test gravity sensitivity, we asked participants to hit a virtual ball rolling down an incline and falling in air, and to report whether ball motion was perceived as natural or unnatural. We manipulated the gravity level independently for the motion on the incline and for the motion in air. The ball was always visible during rolling, whereas it was visible or occluded during falling before interception. The scene included several cues allowing metric calibration of visual space and motion. We found that the perception rate of natural motion was significantly higher and less variable when ball kinematics was congruent with Earth gravity during both rolling and falling. Moreover, the timing of target interception was accurate only in this condition. Neither naturalness perception nor interception timing depended significantly on whether the target was visible during free-fall. Even when occluded, free-fall under natural gravity was correctly extrapolated from the preceding, visible phase of rolling motion. Naturalness perception depended on motor performance, in addition to the gravity level. In sum, both motor and perceptual responses were guided by an internal model of Earth gravity effects. We suggest that, in order to enhance perceptual sensitivity to physical realism, virtual reality should involve visual backgrounds with metric cues and closedloop sensorimotor interactions. This suggestion might be especially relevant for the design of rehabilitation protocols.

Watching the Effects of Gravity. Vestibular Cortex and the Neural Representation of “Visual” Gravity

“…Alike manual interception studies, ocular tracking experiments have shown significantly greater accuracy following target motion modeled according to natural kinematics (gravity and air drag) compared to arbitrary kinematics (hypo- or hypergravity; Diaz et al, 2013a , b ; Delle Monache et al, 2015 , 2019 ; Jörges and López-Moliner, 2019 ; Meso et al, 2020 ). Visual effects of gravity are taken into account, although with variable precision (see below), also in perceptual tasks that do not necessarily involve the production of motor response timed to the target motion, such as the discrimination of motion duration for targets shifting along the vertical (Moscatelli and Lacquaniti, 2011 ; Torok et al, 2019 ; Gallagher et al, 2020 ), time-to-passage estimation during virtual self-motion (Indovina et al, 2013a ), visuomotor synchronization (Zhou et al, 2020 ), naturalness judgments of motion under gravity (La Scaleia et al, 2014 , 2020 ; Ceccarelli et al, 2018 ), speed discrimination of targets moving in different directions (Moscatelli et al, 2019 ), and interpretation of biological motion (Chang and Troje, 2009 ; Maffei et al, 2015 ).…”

“…According to a current hypothesis, an internal model mimics the expected gravity effects on visual targets (Lacquaniti and Maioli, 1989 ; Tresilian, 1997 ; Zago et al, 2008 , 2009 ; Lacquaniti et al, 2013 , 2014 , 2015 ; Jörges and López-Moliner, 2017 ). Compatible with this idea, erroneous expectations of Earth’s gravity effects are evident in the timing of interceptive responses to visual targets moving vertically downward at a constant speed, due to either real weightlessness in a spacelab (McIntyre et al, 2001 ) or simulated weightlessness in the laboratory (Zago et al, 2004 ; Bosco et al, 2012 ; Russo et al, 2017 ; La Scaleia et al, 2020 ). These findings suggest that the brain is able to build an a priori knowledge of gravity effects based on innate mechanisms and/or learning with daily experience.…”

Noise and vestibular perception of passive self-motion

“…Heading direction discrimination is more precise in the horizontal than the vertical plane of the head, and along the naso-occipital and inter-aural axes than intermediate axes ( 108 ). Heading thresholds depend on motion direction in head coordinates (see above) and on body orientation (better performance in the upright orientation), but do not depend on motion direction in world coordinates, demonstrating that the nervous system compensates for gravity ( 108 , 109 ), although incompletely ( 110 ).…”