Embodied virtual reality for the study of real-world motor learning

Abstract

9Background: The motor learning literature focuses on relatively simple laboratory-tasks due to 10 their highly controlled manner and the ease to apply different manipulations to induce learning and 11 adaptation. In recent work we introduced a billiards paradigm and demonstrated the feasibility of 12 real-world neuroscience using wearables for naturalistic full-body motion tracking and mobile 13 brain imaging. Here we developed an embodied virtual reality (VR) environment to our real-world 14 billiards paradigm, which allows us to control the visual feedback for this complex real-world task, 15while maintaining the sense of embodiment. 16 Methods:The setup was validated by comparing real-world ball trajectories with the embodied VR 17 trajectories, calculated by the physics engine. We then ran our real-world learning protocol in the 18 embodied VR. 10 healthy human subjects played repeated trials of the same billiard shot when they 19 held the physical cue and hit a physical ball on the table while seeing it all in VR. 20 Results:We found comparable learning trends in the embodied VR to those we previously reported 21 in the real-world task. 22Conclusions: Embodied VR can be used for learning real-world tasks in a highly controlled VR 23 environment which enables applying visual manipulations, common in laboratory-tasks and in 24 rehabilitation, to a real-world full-body task. Such a setup can be used for rehabilitation, where the 25 use of VR is gaining popularity but the transfer to the real-world is currently limited, presumably, 26 due to the lack of embodiment. The embodied VR enables to manipulate feedback and apply 27 perturbations to isolate and assess interactions between specific motor learning components 28 mechanisms, thus enabling addressing the current questions of motor-learning in real-world tasks. 29 Background 32Motor skill learning is a key feature of our development and our daily lives, from a baby 33 learning to crawl, to an adult learning crafts or sports, or undergoing rehabilitation after an injury 34 or a stroke. It is a complex process, which involves movement in many degrees of freedom (DoF) 35and multiple learning mechanisms. Yet the majority of motor learning literature focuses on simple 36 lab-based tasks with limited DoF such as force-field adaptations [e.g. 1-4], visuomotor 37 perturbations [e.g. 5-9], and sequence-learning of finger tapping or pinching tasks [e.g. 10-13]. 38Real-world neuroscience approach studies neurobehavioral processes in natural behavioral settings 39 [14][15][16][17]. We recently presented a naturalistic real-world motor learning paradigm, using wearables 40 for full body motion tracking and EEG for mobile brain imaging, while making people perform 41 actual real-world tasks, such as playing the competitive sport of pool-table billiards [18,19]. We 42 showed that motor learning is a full body process that involves multiple learning mechanisms, and 43 different subjects might prefer one over the other. 44Now we want to introduce manipulations of real-world t...

Motor learning in real-world pool billiards

“…While lower variability associated with lower means is a well-known statistical phenomenon, it is not evident in all learning tasks. In current billiards paradigm for example, in our VR version of the task, we report no decay in intertrial variability over learning 57 , which may suggest that the learning there is more through www.nature.com/scientificreports/ error-based adaptation. Here, across subjects, we see a decay in intertrial variability which suggests skill learning.…”

“…While the study of real-world tasks takes us closer to understanding real-world motor learning, it is lacking the key advantage of laboratory tasks of highly controlled manipulations of known variables, to isolate specific movement/learning components. We therefore developed an embodied virtual reality (VR) version of our pool task 57 , As the VR-based approach enables visual feedback manipulations while subjects are executing the same real-world task. However, establishing a real-world task as done here is an essential precondition for understanding motor learning and the motor system and the way it has evolved and developed.…”

Brain Activity Reveals Multiple Motor-Learning Mechanisms in a Real-World Task

“…These recent developments highlight the potential for capturing neural signatures of learning in EEG beta. To further validate the current findings, future studies will need to compare the PMBR dynamics during learning of the same paradigm with different dominant mechanism, forced by experimental trickery (i.e., using feedback manipulations and constraints) in laboratory-tasks or real-world task in a virtual reality environment, where feedback manipulations can be applied (Haar et al, 2020 ).…”

Motor learning in real-world pool billiards

“…While the study of real-world tasks takes us closer to understanding real-world motor-learning, it is lacking the key advantage of laboratory tasks (which made them so popular) of highly controlled manipulations of known variables, to isolate specific movement/learning components. To address this issue and introduce manipulations to this real-world task and establish causality, we developed an embodied virtual reality version of our pool task 57 . The VR-based approach can overcome this limitation.…”