Body Motion Analysis for Emotion Recognition in Serious Games

Kaza, Kyriaki; Psaltis, Athanasios; Stefanidis, Kiriakos; Apostolakis, Konstantinos C.; Thermos, Spyridon; Dimitropoulos, Kosmas; Daras, Petros

doi:10.1007/978-3-319-40244-4_4

Cited by 21 publications

(11 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The majority of state of the art emotion recognition frameworks capitalize mainly on facial expression or voice analysis, however, research in the field of experimental and developmental psychology has shown that body movements, body postures, or the quantity or quality of movement behavior in general, can also be of help to differentiate between emotions [57]. To this end, we decided to extract a number of 3D body features, which are deeply inspired by the relevant psychological literature [56] [58].…”

Section: B Body Motion Analysis For Emotion Recognitionmentioning

confidence: 99%

Multimodal Student Engagement Recognition in Prosocial Games

et al. 2018

Self Cite

View full text Add to dashboard Cite

In this paper we address the problem of recognizing student engagement in prosocial games by exploiting engagement cues from different input modalities. Since engagement is a multifaceted phenomenon with different dimensions, i.e., behavioral, cognitive and affective, we propose the modeling of student engagement using real-time data from both the students and the game. More specifically, we apply body motion and facial expression analysis to identify the affective state of students, while we extract features related to their cognitive and behavioral engagement based on the analysis of their interaction with the game. For the automatic recognition of engagement, we adopt a machine learning approach based on artificial neural networks, while for the annotation of the engagement data, we introduce a novel approach based on the use of games with different degrees of challenge in conjunction with a retrospective self-reporting method. To evaluate the proposed methodology, we conducted real-life experiments in four classes, in three primary schools, with 72 students and 144 gameplay recordings in total. Experimental results show the great potential of the proposed methodology, which improves the classification accuracy of the three distinct dimensions with a detection rate of 85%. A detailed analysis of the role of each component of the Game Engagement Questionnaire (GEQ), i.e., immersion, presence, flow and absorption, in the classification process is also presented in this paper.

show abstract

Section: B Body Motion Analysis For Emotion Recognitionmentioning

confidence: 99%

Multimodal Student Engagement Recognition in Prosocial Games

et al. 2018

Self Cite

View full text Add to dashboard Cite

show abstract

“…In order to compare the proposed method with other classification methods, we calculated the most commonly used features, such as kinematic related features (velocity, acceleration, kinetic energy), spatial extent related features (bounding box volume, contraction index, density), smoothness related features, leaning related features and distance related features. During features extraction we strictly followed approach presented in Reference [ 23 ], since the authors obtained very promising results on a database derived from Kinect recordings. We juxtaposed several well known classification methods to verify the above-mentioned features and their effectiveness in gestures-based emotion recognition.…”

Section: Resultsmentioning

confidence: 99%

“…However, it must be emphasised that the superior performance is associated with the type of analysed data. In Reference [ 23 ] emotions are represented as predetermined gestures (each emotion is assigned to particular type of gesture, for example, power pose to happiness). The actors/actresses are instructed how to present particular emotional state prior to recording.…”

Section: Introductionmentioning

confidence: 99%

Emotion Recognition from Skeletal Movements

Sapiński

Kamińska

Pelikant

et al. 2019

Entropy

View full text Add to dashboard Cite

Automatic emotion recognition has become an important trend in many artificial intelligence (AI) based applications and has been widely explored in recent years. Most research in the area of automated emotion recognition is based on facial expressions or speech signals. Although the influence of the emotional state on body movements is undeniable, this source of expression is still underestimated in automatic analysis. In this paper, we propose a novel method to recognise seven basic emotional states—namely, happy, sad, surprise, fear, anger, disgust and neutral—utilising body movement. We analyse motion capture data under seven basic emotional states recorded by professional actor/actresses using Microsoft Kinect v2 sensor. We propose a new representation of affective movements, based on sequences of body joints. The proposed algorithm creates a sequential model of affective movement based on low level features inferred from the spacial location and the orientation of joints within the tracked skeleton. In the experimental results, different deep neural networks were employed and compared to recognise the emotional state of the acquired motion sequences. The experimental results conducted in this work show the feasibility of automatic emotion recognition from sequences of body gestures, which can serve as an additional source of information in multimodal emotion recognition.

show abstract

“…A series of multimodal observation channels are established from input sensors connected to player devices including microphones, cameras and keyboard, in addition to more advanced gaming sensors such as Microsoft Kinect and Leap Motion. Using sensing and classification techniques emotion from voice, facial expression and body language is acquired and then fusion processes applied to provide the emotional state of the user [5][6].…”

Section: Interaction With Sensorsmentioning

confidence: 99%

ProsocialLearn: A Prosocial Games Marketplace

D'Andria¹,

Garrido²,

Boniface

et al. 2018

2018 9th IFIP International Conference on New Technologies, Mobility and Security (NTMS)

Self Cite

View full text Add to dashboard Cite

Abstract-ProsocialLearn is a digital pro-social games platform. The ProsocialLearn project has delivered a series of disruptive innovations for the production and distribution of prosocial digital games that engage children (7-10 years old). Additionally it has stimulated technology transfer from the games industry to the educational sector. ProsocialLearn fosters the creation of a new market for digital games aimed at increasing social inclusion and academic performance, as well as a distribution channel to deliver prosocial games to children and teachers in European schools. Furthermore, it provides a proven pro-social methodology to design digital games. The ProsocialLearn platform makes available a series of APIs, which game developers use to integrate many of the ProsocialLearn functions into games, i.e. emotion and engagement monitoring, in-game achievements, games adaptation based on Prosocial Learning Objectives (PLOs), and micro-transactions.

show abstract

Body Motion Analysis for Emotion Recognition in Serious Games

Cited by 21 publications

References 15 publications

Multimodal Student Engagement Recognition in Prosocial Games

Multimodal Student Engagement Recognition in Prosocial Games

Emotion Recognition from Skeletal Movements

ProsocialLearn: A Prosocial Games Marketplace

Contact Info

Product

Resources

About