Adaptive Rehabilitation Bots in Serious Games

Abstract

In recent years, we have witnessed a growing adoption of serious games in telerehabilitation by taking advantage of advanced multimedia technologies such as motion capture and virtual reality devices. Current serious game solutions for telerehabilitation suffer form lack of personalization and adaptiveness to patients’ needs and performance. This paper introduces “RehaBot”, a framework for adaptive generation of personalized serious games in the context of remote rehabilitation, using 3D motion tracking and virtual reality environments. A personalized and versatile gaming platform with embedded virtual assistants, called “Rehab bots”, is created. Utilizing these rehab bots, all workout session scenes will include a guide with various sets of motions to direct patients towards performing the prescribed exercises correctly. Furthermore, the rehab bots employ a robust technique to adjust the workout difficulty level in real-time to match the patients’ performance. This technique correlates and matches the patterns of the precalculated motions with patients’ motions to produce a highly engaging gamified workout experience. Moreover, multimodal insights are passed to the users pointing out the joints that did not perform as anticipated along with suggestions to improve the current performance. A clinical study was conducted on patients dealing with chronic neck pain to prove the usability and effectiveness of our adjunctive online physiotherapy solution. Ten participants used the serious gaming platform, while four participants performed the traditional procedure with an active program for neck pain relief, for two weeks (10 min, 10 sessions/2 weeks). Feasibility and user experience measures were collected, and the results of experiments show that patients found our game-based adaptive solution engaging and effective, and most of them could achieve high accuracy in performing the personalized prescribed therapies.

A systematic review on the usability of robotic and virtual reality devices in neuromotor rehabilitation: patients’ and healthcare professionals’ perspective

“…The full report of the information of the included studies is presented in the synoptic table (Additional File 1 : Appendix 1 ) [ 39 – 106 ].…”

“…Patients referred good acceptability regardless the different levels of immersion. Most participated in non-immersive VR-based therapy [ 40 , 44 – 46 , 48 , 50 , 52 , 53 , 57 – 62 , 64 – 73 , 75 – 77 ], whereas fully-immersive devices (i.e., HMD) were used in six studies [ 39 , 42 , 43 , 54 , 63 , 78 ]. Semi-immersive VR (i.e., Kinect with large TV or projector screens) was tested in five studies [ 47 , 49 , 51 , 55 , 57 , 74 ].…”

VR-PEER: A Personalized Exer-Game Platform Based on Emotion Recognition

“…The difficulty of the exergame changed dynamically according to the score obtained at a given time. The Rehab Bots game was created in [24]. It is based on the virtual assistant identified by therapists and contains three main levels that are: (1) a virtual assistance robot that shows patients how to properly perform the exercises, (2) an intelligent movement notifications module that analyzes and represents patients in 3D, then, if necessary, adjusts the game to the appropriate level to achieve the best outcome, and (3) a dynamic correction module that takes into account both the level of game complexity and the virtual assistant readings in order to create a series of exercises that are more suited to the patients' capacities.…”

“…We notice from Table 1 that there are studies that have developed serious games for rehabilitation, but they are not adapted to the patient [3,40]. In the same context, if they are adapted they are personalized but not intended for motor rehabilitation [50], or the adaptation is only at the beginning of the exercise, regardless of what happens during it [17,19,24]. The previous works may be oriented to kinesthetic rehabilitation and adapted dynamically to the game's surroundings, but they do not take into account the virtual reality aspect [46,49].…”

“…In this regard, the personalized exer-game addresses this issue. Several studies in the literature focused on such games, some of them are interested in static adaptation [21,22], which makes changes only at the beginning of the game, as opposed to dynamic adaptation, which allows you to adapt the game during the long period of play [17,[22][23][24][25][26][27]. Even if the adaptation is dynamic, it is according to parameters that do not really express the situation of the patient such as the score, or other systems that rely mainly on specialists to intervene in every adaptation process [20,24].…”