VR-PEER: A Personalized Exer-Game Platform Based on Emotion Recognition

Abstract

Motor rehabilitation exercises require recurrent repetitions to enhance patients’ gestures. However, these repetitive gestures usually decrease the patients’ motivation and stress them. Virtual Reality (VR) exer-games (serious games in general) could be an alternative solution to address the problem. This innovative technology encourages patients to train different gestures with less effort since they are totally immersed in an easy to play exer-game. Despite this evolution, patients, with available exer-games, still suffer in performing their gestures correctly without pain. The developed applications do not consider the patients psychological states when playing an exer-game. Therefore, we believe that is necessary to develop personalized and adaptive exer-games that take into consideration the patients’ emotions during rehabilitation exercises. This paper proposed a VR-PEER adaptive exer-game system based on emotion recognition. The platform contain three main modules: (1) computing and interpretation module, (2) emotion recognition module, (3) adaptation module. Furthermore, a virtual reality-based serious game is developed as a case study, that uses updated facial expression data and provides dynamically the patient’s appropriate game to play during rehabilitation exercises. An experimental study has been conducted on fifteen subjects who expressed the usefulness of the proposed system in motor rehabilitation process.

Assessing the effectiveness of virtual reality serious games in post-stroke rehabilitation: a novel evaluation method

“…B.Recognition: After data preprocessing, we employed the method proposed in [50] to classify the patient's facial expressions into various emotion categories, including positive, negative, neutral (figure 4). Our previous work [7] involved the use of the combination of MoodMe Emotions Barracuda SDK [49] for pre-processing, and DeepFace [48], for emotion classification, DeepFace a neural network model developed by Facebook's AI Research team in 2014. The recognition module (MoodMe + Deep-Face) has demonstrated its capabilities in our previous research [7].…”

“…Our previous work [7] involved the use of the combination of MoodMe Emotions Barracuda SDK [49] for pre-processing, and DeepFace [48], for emotion classification, DeepFace a neural network model developed by Facebook's AI Research team in 2014. The recognition module (MoodMe + Deep-Face) has demonstrated its capabilities in our previous research [7]. In this paper, our emphasis is on using emotion recognition to estimate the level of motivation in patients during serious games.…”

“…Positive emotions during therapy and patient motivation are crucial for successful rehabilitation and facilitating recovery after a stroke, emphasizing that positive treatment outcomes depend not only on the efforts of healthcare professionals but also on the willingness, active participation, and motivation of the patient. In contrast, Research has shown that positive emotions lead to positive therapeutic outcomes, while negative emotions can result in unfavorable outcomes [4][5][6][7]; Low motivation in stroke patients has been shown to increase the difficulty of rehabilitation [8,9]. To assist this large number of stroke victims, new therapy methods such as virtual reality (VR) and interactive video games (IVG) have been developed [10][11][12].…”

The Impact of Emotions on the Speed and Score of Computer Games

“…For instance, Valve Software has actively experimented with biometrics by including them in a special build of Left 4 Dead 2. Valve Software sees affect emotion as a crucial component of future games [15]. In the literature review, several methods used for facial expression detection are identified, and the existing systems are contrasted.…”

A System of Emotion Recognition and Judgment and Its Application in Adaptive Interactive Game

“…Granato et al [ 19 ] predicted the subjects’ emotions during video game sessions and indicated that the obtained results could improve the game design. Izountar et al [ 20 ] proposed a VR-PEER adaptive exergame system and developed a virtual reality-based serious game as a case study. The test results showed that fifteen participants expressed the usefulness of the system in motor rehabilitation processes.…”