Visuo-haptic Illusions for Motor Skill Acquisition in Virtual Reality

Abstract

In this article we investigate the potential of using visuo-haptic illusions in Virtual Reality environment to learn motor skills in a real environment. We report on an empirical study where 20 participants perform a multi-object pick-and-place task. The results show that although users do not perform the same motion trajectories in the virtual and real environments, skills acquired in VR augmented with visuo-haptic illusions can be successfully reused in a real environment: There is a high amount of skill transfer (78.5%), similar to the one obtained in an optimal real training environment (82.4%); Finally, participants did not notice the illusion and were enthusiastic about the VR environment. Our findings invite designers and researchers to consider visuo-haptic illusions to help operators to learn motor skills in a cost-effective environment. CCS CONCEPTS• Software and its engineering → Virtual worlds training simulations; • Hardware → Tactile and hand-based interfaces;• Human-centered computing → Empirical studies in HCI.

Generalizing the Hand Redirection Function in Virtual Reality

“…In the ballistic phase, the movement is faster as the users do not rely on vision to correct errors. Recent studies [2,14] about visuo-haptic illusions show that some participants strongly adjust their trajectory of their real hand towards the target only during the final part of the movement. This observation is in line with the two-component model suggesting that participants only noticed the offset between the predicted and actual position of the virtual hand at the end of the ballistic movement.…”

“…Visuo-haptic illusions exploit visual dominance to enrich user immersion in virtual reality (VR). For example, users can interact with many virtual objects despite a limited number of props (physical objects) present in the physical environment [14,22]. It is also possible to modify an object shape [1,8] or its physical properties like weight [9,20,30] or stiffness [25].…”

Généralisation de la Fonction de Redirection de la Main en Réalité Virtuelle

“…Ce résultat semble surprenant de prime abord car il suggère que la détection de l'illusion se fait à l'échelle de l'essai, et non à l'échelle des ajustements moteurs pendant l'essai. En effet, des travaux précédents [2,14] ont mis en évidence une correction à la fin du mouvement dans la trajectoire de la main réelle suggérant que l'utilisateur réalise d'abord un mouvement balistique, puis s'aperçoit du décalage entre la position prédite et la position perçue pour finalement ajuster son mouvement. Une de nos hypothèses était que la détection de l'illusion se passait juste avant cette phase d'ajustement.…”

“…Ce décalage permet lorsque la main virtuelle atteint l'objet virtuel, que la main réelle atteigne l'objet physique correspondant. Cette cohérence sensorielle entre ce que voit l'utilisateur et ce qu'il touche permet par exemple d'interagir avec de nombreux objets virtuels bien qu'un nombre limité de props (objet physiques) soient présents dans l'environnement physique [14,22]. Cependant lorsque le décalage entre la main réelle et virtuelle est trop important, il y a une incohérence sensorielle entre ce qui est perçu par la vision et par la proprioception.…”

Study of the Impact of Visual and Haptic Sensory Sensitivities on the Detection of Visuo-Haptic Illusions in Virtual Reality

“…The brain rejects an illusion if it notices that something is wrong. Several studies aimed to estimate the maximum amplitude beyond which participants (population level) notice it or judge it uncomfortable [1,7,8,21,23,30,44,51,69] However, these detection thresholds are low and can be increased by customizing the implementation of illusions to each participant's characteristics (individual level). Lebrun et al [37] show that it is possible, after a short calibration phase, to adjust the amplitude of hand redirection for each participant and thus increase the detection threshold in comparison with the one obtained at the population level.…”