Using body movement and posture for emotion detection in non-acted scenarios

Abstract

In this paper, we explored the use of features that represent body posture and movement for automatically detecting people's emotions in non-acted standing scenarios. We focused on four emotions that are often observed when people are playing video games: triumph, frustration, defeat, and concentration. The dataset consists of recordings of the rotation angles of the player's joints while playing Wii sports games. We applied various machine learning techniques and bagged them for prediction. When body pose and movement features are used we can reach an overall accuracy of 66.5% for differentiating between these four emotions. In contrast, when using the raw joint rotations, limb rotation movement, or posture features alone, we were only able to achieve accuracy rates of 59%, 61%, and 62% respectively. Our results suggest that features representing changes in body posture can yield improved classification rates over using static postures or joint information alone.

Improved Deep Feature Learning by Synchronization Measurements for Multi-Channel EEG Emotion Recognition

“…Transitions between emotion states are accompanied by complex neural processes and physiological changes. In addition to facial expressions [2], speech [3][4][5], and body movement [6,7], electrophysiological signals and endocrine-related indicators can reflect changes in emotion states [8][9][10] as well. However, these same physical characteristics are easily affected by the subjective will of a person as well as the external environment.…”

Modelling Player Preferences in AR Mobile Games

“…We use player movement to predict preferences. Related work has been conducted in [8] and [9] in detecting emotions from movement data of people playing Nintendo Wii games. In [9], a different approach to predicting affective states 3 is followed.…”

“…In [9], a different approach to predicting affective states 3 is followed. Our approach is similar to [8], which predicts distinct emotional states (Triumph, Concentration, Defeat and Frustration were the states explored in the study). The 2 studies are similar, as ground truth for predictions is multiple observer agreement.…”

A mobile application to report and detect 3D body emotional poses

“…Saneiro et al (2014) developed a system for tagging body movements with emotions providing information which can be used with data mining techniques. In addition, Garber- Barron and Si (2012) found that body postures after changes were more representative for the automatic detection of emotions than still body poses. Some authors have proposed automatic techniques for classifying two-dimensional (2D) static images into a set of emotional states (Schindler et al, 2008;De Silva and Bianchi-Berthouze, 2004), starting a challenging research line in the field of affective computing.…”

“…Existing automatic emotion detection mechanisms from body poses mainly use computer vision techniques in which relevant information is extracted from images (see an example in (De Silva and Bianchi-Berthouze, 2004)) or videos (such as (Garber- Barron and Si, 2012)). This visual information is normally taken from either independent cameras (De Silva and Bianchi-Berthouze, 2004) or cameras integrated into consoles such as the Microsoft Xbox (Rázuri et al, 2015).…”