Automatic Recognition of Affective Body Movement in a Video Game Scenario

Abstract

Abstract. This study aims at recognizing the affective states of players from non-acted, non-repeated body movements in the context of a video game scenario. A motion capture system was used to collect the movements of the participants while playing a Nintendo Wii tennis game. Then, a combination of body movement features along with a machine learning technique was used in order to automatically recognize emotional states from body movements. Our system was then tested for its ability to generalize to new participants and to new body motion data using a sub-sampling validation technique. To train and evaluate our system, online evaluation surveys were created using the body movements collected from the motion capture system and human observers were recruited to classify them into affective categories. The results showed that observer agreement levels are above chance level and the automatic recognition system achieved recognition rates comparable to the observers' benchmark.

Online feedback system for public speakers

“…Most systems take input data from motion capture equipment, such as the VICON system [17] or gaming devices like Nintendo Wii [20] because these devices could provide great details about body motion. The most noticeable work in gestural emotion detection has been done by Camurri and the Infomus Lab [21].…”

Game User Experience Evaluation

“…Even though in this study we were yet unable to automatically perform such an analysis, new tools for gesture and affective movements detection are becoming available, and a motion capture system could facilitate the capturing of these different types of behaviors automatically. Berthouze and colleagues (Bianchi-Berthouze and Kleinsmith 2003;Kleinsmith et al 2011;Savva et al 2012) proposed a low-level description of body posture and movement that enable the mapping of bodily expressions into emotion categories or emotion dimensions. By using low-level descriptions of posture, motion capture, and connectionist or statistical modeling techniques to these descriptions, they have suggested that mapping models can easily be adapted to detect different types of expressions irrespective of the context in which these expressions are displayed.…”

Body Motion Analysis for Emotion Recognition in Serious Games

“…This method achieved a 61.3 % recognition rate when evaluated at a dataset of recorded body movements of actors who were asked to express freely the six basic emotions. Savva et al [13] proposed an automatic recognition method of affective body movement in the context of a Nintendo Wii tennis game which feeds dynamic movement features to a Recurrent Neural Network (RNN) algorithm. This method was tested at a dataset of nonacted movements captured with Animazoo IGS-190 during gameplay and reached a recognition rate of 57.46 %, comparable with the 61.49 % accuracy of human observers' recognition.…”