Adapting Virtual Embodiment Through Reinforcement Learning

Porssut, Thibault; Hou, Yufei; Blanke, Olaf; Herbelin, Bruno; Boulic, Ronan

doi:10.1109/tvcg.2021.3057797

Cited by 11 publications

(16 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[ 45 , 46 ] In this way, the CNS can activate a physiological learning mechanism called “reinforcement learning” which implies an increase in the specific information of a movement to produce an effective improvement in performance quality. [ 47 ] This boosting in neural plasticity with a consequent potentiation in the functional outcomes could be why our patients achieved higher results after this novel approach than following the conventional training.…”

Section: Discussionmentioning

confidence: 92%

“…Indeed, VRRS is designed to put the patient in a situation that generates increased feedback to his/her CNS (augmented feedback) through exercises performed in a virtual environment helping to develop the knowledge of the results of movements (knowledge of the results) and the knowledge of the quality of movements (knowledge of performance), leading to a training-specific motor learning [45,46] . In this way, the CNS can activate a physiological learning mechanism called “reinforcement learning” which implies an increase in the specific information of a movement to produce an effective improvement in performance quality [47] . This boosting in neural plasticity with a consequent potentiation in the functional outcomes could be why our patients achieved higher results after this novel approach than following the conventional training.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Advances in neuroRehabilitation of TREM2-related dementia

Luca¹,

Marra²,

Pollicino³

et al. 2022

Medicine

View full text Add to dashboard Cite

Rationable:The aim of this study is to investigate the effects of an advanced neuroRehabilitation protocol using virtual reality in the treatment of a patient with fronto-temporal dementia due to TREM2 mutation.Patient's concern: A 41-year-old caucasian male, affected by Nasu-Hakola Disease (NHD), presented a 1-year history of change in behavioral and cognitive functioning, before our observation. The onset of the disease was characterized by severe pain in the lower limbs and knees with limitations in the performance of daily life activities.Diagnosis: Motor and cognitive deficits in NHD.Interventions: As the patient was in a chronic phase, to manage his cognitive and motor status, we decided to treat him by using a specific rehabilitation protocol, including 2 different types of training: conventional cognitive and motor treatment and a combined advanced approach using the virtual reality rehabilitation system (VRRS). The two protocols were separated by 4 weeks of rest, to avoid/reduce a cumulative effect. The patient's cognitive and motor profile was assessed three times: that is before (at T0) and after (at T1) the conventional training as well as at the beginning (T2) and at the end of the combined experimental approach (T3).Outcomes: After the combined therapeutic approach with the virtual reality rehabilitation system, we observed a significant reduction in anxiety, apathy, indifference and depressive symptoms with a more evident motor improvement involving the head and the trunk control.Lessons: Virtual reality can be considered a promising tool for the motor and cognitive rehabilitation of rare neurodegenerative disorders, including patients with NHD.Abbreviations: CCT = conventional cognitive training, CNS = central nervous system, CT = conventional training, FTD = frontotemporal dementia, NHD = Nasu-Hakola disease, RCI = reliable change index, SCT-VRRS = standard cognitive training-virtual reality rehabilitation system, SPT = standard physiotherapy, VR = virtual reality, VRRS = virtual reality rehabilitation system.

show abstract

Section: Discussionmentioning

confidence: 92%

Section: Discussionmentioning

confidence: 99%

Advances in neuroRehabilitation of TREM2-related dementia

Luca¹,

Marra²,

Pollicino³

et al. 2022

Medicine

View full text Add to dashboard Cite

show abstract

“…Despite the fact that some distortions can be tolerated without breaking the sense of embodiment, when one of its component is strongly disrupted, a break in embodiment occurs, which should be avoided for an optimal experience in VR (Kokkinara and Slater, 2014;Porssut et al, 2019;Porssut et al, 2022a;Porssut et al, 2022b). For example, the avatar's movements can be distorted without breaking the sense of agency (Farrer et al, 2008;Kokkinara et al, 2015;Kasahara et al, 2017;Rietzler et al, 2017;Galvan Debarba et al, 2018;Porssut et al, 2019;Porssut et al, 2021) and can, therefore, be used to improve, for example, interactions with 3D objects in VR (Burns et al, 2005;Azmandian et al, 2016) or navigation (Nguyen et al, 2020).…”

Section: Related Workmentioning

confidence: 99%

Unintentional synchronization with self-avatar for upper- and lower-body movements

Boban

Strauss²,

Decroix³

et al. 2023

Front. Virtual Real.

Self Cite

View full text Add to dashboard Cite

The subjective experience of embodying an avatar when immersed in virtual reality (VR) is known to support the sense of presence and to help with the interaction in a virtual environment. Virtual embodiment is often thought of as the consequence of replacement of the physical body by a virtual one, with a sense of agency for the avatar obtained by making the avatar’s body follow the user’s movements. This unidirectional motor link was, however, challenged by studies observing the opposite effect under different circumstances, for example, in a slow-motion context or when an arm movement was snapped on a predefined axis. These reports are, however, still rare or anecdotal. With the idea of a generalized bidirectional relationship between the user and the avatar in mind, we established a methodology to systematically provoke and study the circumstances under which participants follow the movements of their avatar during long repetitive movements without having been instructed to do so. A preliminary study confirmed that our virtual experimental setup, using full-body motion capture, avatar animation, and virtual mirrors, supports a strong sense of agency and body ownership for the avatar while enabling the experimental manipulation of the avatar’s movement. In the main experimental study, where participants performed repetitive upper- and lower-body movements while their avatar animations were either congruent or out-of-phase, we observed that almost all participants synchronized with their avatar at least once, for ∼47% of trials for lower limb movements and ∼38% for upper limb movements. Participants still reported low agency and ownership for the avatar under the incongruent condition, but, most interestingly, some of them also reported that their movements were not influenced by the avatar despite the behavioral effect. Our methodological approach and results contribute to the characterization of the conditions of occurrence of the self-avatar follower effect and, thereby, to identifying an enriched interaction design for VR, involving complex avatar–user mutual interdependencies.

show abstract

“…It has been shown that a continuous distortion introducing a spatial discrepancy between the real (hidden) and the virtual (visible) arm in a reaching task is rather well-tolerated [6,20,21]. More specifically, participants still report being the agent performing the action despite a relatively large distortion, typically when it helps them to reach a goal (around +2dB change in movement's speed in the study from Debarba et al [20]) as opposed to when it prevents them from doing so.…”

Section: Introductionmentioning

confidence: 99%

Avatar error in your favor: Embodied avatars can fix users’ mistakes without them noticing

Delahaye

Blanke²,

Boulic³

et al. 2023

PLoS ONE

View full text Add to dashboard Cite

In immersive Virtual Reality (VR), users can experience the subjective feeling of embodiment for the avatar representing them in a virtual world. This is known to be strongly supported by a high Sense of Agency (SoA) for the movements of the avatar that follows the user. In general, users do not self-attribute actions of their avatar that are different from the one they actually performed. The situation is less clear when actions of the avatar satisfies the intention of the user despite distortions and noticeable differences between user and avatar movements. Here, a within-subject experiment was condutected to determine wether a finger swap helping users to achieve a task would be more tolerated than one penalizing them. In particular, in a context of fast-paced finger movements and with clear correct or incorrect responses, we swapped the finger animation of the avatar (e.g. user moves the index finger, the avatar moves the middle one) to either automatically correct for spontaneous mistakes or to introduce incorrect responses. Subjects playing a VR game were asked to report when they noticed the introduction of a finger swap. Results based on 3256 trials (∼24% of swaps noticed) show that swaps helping users have significantly fewer odds of being noticed (and with higher confidence) than the ones penalizing users. This demonstrates how the context and the intention for motor action are important factors for the SoA and for embodiment, opening new perspectives on how to design and study interactions in immersive VR.

show abstract

Adapting Virtual Embodiment Through Reinforcement Learning

Cited by 11 publications

References 55 publications

Advances in neuroRehabilitation of TREM2-related dementia

Advances in neuroRehabilitation of TREM2-related dementia

Unintentional synchronization with self-avatar for upper- and lower-body movements

Avatar error in your favor: Embodied avatars can fix users’ mistakes without them noticing

Contact Info

Product

Resources

About