So Predictable! Continuous 3D Hand Trajectory Prediction in Virtual Reality

Gamage, Nisal Menuka; Ishtaweera, Deepana; Weigel, Martin; Withana, Anusha

doi:10.1145/3472749.3474753

Cited by 24 publications

(8 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the context of VR, an example is the work of Gamage et al (2021). Who used regression models to predict the path of the hands in VR and real-time.…”

Section: State Of the Artmentioning

confidence: 99%

Long short-term memory prediction of user’s locomotion in virtual reality

Mayor,

Calleja,

Fuentes-Hurtado

2024

Virtual Reality

View full text Add to dashboard Cite

Nowadays, there is still a challenge in virtual reality to obtain an accurate displacement prediction of the user. This could be a future key element to apply in the so-called redirected walking methods. Meanwhile, deep learning provides us with new tools to reach greater achievements in this type of prediction. Specifically, long short-term memory recurrent neural networks obtained promising results recently. This gives us clues to continue researching in this line to predict virtual reality user’s displacement. This manuscript focuses on the collection of positional data and a subsequent new way to train a deep learning model to obtain more accurate predictions. The data were collected with 44 participants and it has been analyzed with different existing prediction algorithms. The best results were obtained with a new idea, the use of rotation quaternions and the three dimensions to train the previously existing models. The authors strongly believe that there is still much room for improvement in this research area by means of the usage of new deep learning models.

show abstract

“…In the context of VR, an example is the work of Gamage et al (2021). Who used regression models to predict the path of the hands in VR and real-time.…”

Section: State Of the Artmentioning

confidence: 99%

Long short-term memory prediction of user’s locomotion in virtual reality

Mayor,

Calleja,

Fuentes-Hurtado

2024

Virtual Reality

View full text Add to dashboard Cite

show abstract

“…In the context of VR, an example is the work of Gamage et al [22]. Who used regression models to predict the path of the hands in VR and real time.…”

Section: State Of the Artmentioning

confidence: 99%

Long-short term memory prediction of user’s locomotion in Virtual Reality

Mayor,

Calleja,

Fuentes

et al. 2023

Preprint

View full text Add to dashboard Cite

Virtual reality has some well-known limitations when the user walks naturally. A user experiencing virtual reality may collide with the limits of the physical workspace. To solve this, there exist techniques, such as the redirected walking methods, that try to trick the human brain to avoid these limits. However, these methods require good predictions of where users want to go in the future. On the other hand, Deep learning is helping in so many areas to make accurate predictions. Specifically, Long short-term memory obtained promising results in this virtual reality prediction task giving us clues to continue researching in this line. This manuscript presents an experiment carried out with 44 participants collecting positional data. These data have been used to test different existing prediction algorithms. In addition, a way to improve the previous results obtained with deep learning is presented. These results are achieved with the use of rotation quaternions and various positional data in three dimensions. The authors strongly believe that there is still much room for improvement in this research area.

show abstract

“…In the context of rehabilitation, it can be leveraged to create personalized rehabilitation programs for individuals with mobility impairments, such as those caused by stroke or spinal cord injury. Additionally, it can enable natural and intuitive control of devices, such as virtual reality systems, through foot gestures in human-machine interaction [2]. In the field of robotics, anticipating the next foot placement in advance can help enhance the stability and robustness of bipedal robot control in changing environments [3].…”

Section: List Of Abbreviationsmentioning

confidence: 99%

A Deep Learning-Based Approach for Foot Placement Prediction

Lee

Asbeck

2023

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Foot placement prediction can be important for exoskeleton and prosthesis controllers, human-robot interaction, or body-worn systems to prevent slips or trips. Previous studies investigating foot placement prediction have been limited to predicting foot placement during the swing phase, and do not fully consider contextual information such as the preceding step or the stance phase before push-off. In this study, a deep learning-based foot placement prediction approach was proposed, where the deep learning models were designed to sequentially process data from three IMU sensors mounted on pelvis and feet. The raw sensor data are pre-processed to generate multi-variable time-series data for training two deep learning models, where the first model estimates the gait progression and the second model subsequently predicts the next foot placement. The ground truth gait phase data and foot placement data are acquired from a motion capture system. Ten healthy subjects were invited to walk naturally at different speeds on a treadmill. In cross-subject learning, the trained models had a mean distance error of 5.93 cm for foot placement prediction. In single-subject learning, the prediction accuracy improved with additional training data, and a mean distance error of 2.60 cm was achieved by fine-tuning the cross-subject validated models with the target subject data. Even from 25-81% in the gait cycle, mean distance errors were only 6.99 cm and 3.22 cm for cross-subject learning and single-subject learning, respectively.

show abstract

So Predictable! Continuous 3D Hand Trajectory Prediction in Virtual Reality

Cited by 24 publications

References 51 publications

Long short-term memory prediction of user’s locomotion in virtual reality

Long short-term memory prediction of user’s locomotion in virtual reality

Long-short term memory prediction of user’s locomotion in Virtual Reality

A Deep Learning-Based Approach for Foot Placement Prediction

Contact Info

Product

Resources

About