Sensing Affective States Using Facial Expression Analysis

Samara, Anas; Galway, Leo; Bond, Raymond; Wang, Hui

doi:10.1007/978-3-319-48746-5_35

Cited by 10 publications

(8 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…A common classification approach is based on distances and angles of landmarks. Samara et al [24] use the Euclidean International Journal of Computer Games Technology distance among face points to train a Support Vector Machine (SVM) model to detect expressions. Similarly Chang et al [25] use 12 distances calculated from 14 landmarks to detect fear, love, joy, and surprise.…”

Section: Related Workmentioning

confidence: 99%

“…Previous work focuses on detecting facial expressions per se, including the six universal facial expressions of emotion, typically reporting accuracy rates of machine learning models used to detect those predefined facial expressions. A significant number of those approaches train the models using datasets with images and videos of actors performing facial expressions [24,[26][27][28]46], subjects watching video clips [25,29,38], or subjects undergoing social exposure [38]. As previously mentioned, those are artificial situations that are significantly different from an interaction with a game.…”

Section: Comparison With Previous Workmentioning

confidence: 99%

See 1 more Smart Citation

Automated Analysis of Facial Cues from Videos as a Potential Method for Differentiating Stress and Boredom of Players in Games

Bevilacqua

Engström

Backlund

2018

International Journal of Computer Games Technology

View full text Add to dashboard Cite

Facial analysis is a promising approach to detect emotions of players unobtrusively; however approaches are commonly evaluated in contexts not related to games or facial cues are derived from models not designed for analysis of emotions during interactions with games. We present a method for automated analysis of facial cues from videos as a potential tool for detecting stress and boredom of players behaving naturally while playing games. Computer vision is used to automatically and unobtrusively extract 7 facial features aimed at detecting the activity of a set of facial muscles. Features are mainly based on the Euclidean distance of facial landmarks and do not rely on predefined facial expressions, training of a model, or the use of facial standards. An empirical evaluation was conducted on video recordings of an experiment involving games as emotion elicitation sources. Results show statistically significant differences in the values of facial features during boring and stressful periods of gameplay for 5 of the 7 features. We believe our approach is more user-tailored, convenient, and better suited for contexts involving games.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Comparison With Previous Workmentioning

confidence: 99%

Automated Analysis of Facial Cues from Videos as a Potential Method for Differentiating Stress and Boredom of Players in Games

Bevilacqua

Engström

Backlund

2018

International Journal of Computer Games Technology

View full text Add to dashboard Cite

show abstract

“…However, one can view these percentages differently by considering the fact that some expressions are much more precisely recognised than others. Generally, detecting states such as happy and surprise is comparably superior than detecting other states such as contempt, neutral, fear, angry, sadness and disgust, which is possibly due to the similarity in the geometric shape of these expressions (Samara et al 2016). Moreover, the work presented in (Joho et al 2009) underlined this type of grouping, by devising the pronounce level of the associated expressions, where these expressions belong to a low pronounced level.…”

Section: Expression Classification During Humancomputer Interactionmentioning

confidence: 99%

“…Subsequently, resultant features are represented by finding the Euclidean distances among all facial landmark points. Consequently, the facial expression is finally represented as a 1176-dimension feature vector, resulting from 49 Cartesian coordinate combinations (Samara et al 2016). …”

Section: Feature Extraction and Distance-based Representationmentioning

confidence: 99%

Affective state detection via facial expression analysis within a human–computer interaction context

Samara

Galway

Bond

et al. 2017

J Ambient Intell Human Comput

Self Cite

View full text Add to dashboard Cite

The advancement in technology indicates that there is an opportunity to enhance human-computer interaction by way of affective state recognition. Affective state recognition is typically based on passive stimuli such as watching video clips, which does not reflect genuine interaction. This paper presents a study on affective state recognition using active stimuli, i.e. facial expressions of users when they attempt computerised tasks, particularly across typical usage of computer systems. A data collection experiment is presented for acquiring data from normal users whilst they interact with software, attempting to complete a set of predefined tasks. In addition, a hierarchical machine learning approach is presented for facial expression-based affective state recognition, which employs an Euclidean distance-based feature representation, conjointly with a customised encoding for users' self-reported affective states. Consequently, the aim is to find the potential relationship between the facial expressions, as defined by Paul Ekman, and the self-reported emotional states specified by users using Russells Circumplex model, in relation to the actual feelings and affective states. The main findings of this study suggest that facial expressions cannot precisely reveal the actual feelings of users whilst interacting with common computerised tasks. Moreover, during active interaction tasks more variation occurs within the facial expressions of participants than occurs within passive interaction.

show abstract

“…However, recognising User Perplexity and Confusion is still non-trivial and a challenging objective, particularly in the context of HCI. Additionally, the complexity of human emotions and the peculiarity of the relationship between humans and machines provides a challenge in Affective Computing and relevant research themes (Samara et al, 2016(Samara et al, , 2017.…”

Section: User Perplexity In Human-computer Interactionmentioning

confidence: 99%

Adaptive User Experience Based on Detecting User Perplexity

Samara

Bond

Galway

et al. 2018

Electronic Workshops in Computing

Self Cite

View full text Add to dashboard Cite

As humans, our abilities and performance while carrying out different tasks differ due to several factors, such as health conditions, mental processing capabilities, emotional feelings, task difficulty and the desire to achieve it. Additionally, understanding human feelings and states that may entail information about achieving certain tasks can be an intricate job. Subsequently, this paper aims to outline the main aspects of self-ware systems and adaptive Human-Computer Interaction styles. There is an opportunity to enable machines to be more perceptive devices that can recognise innate human factors, which could be used to assist and improve the human's performance and effectiveness. This paper shows use-case scenarios of intelligent and affect-aware systems as well as presenting a conceptual model for User Perplexity as an important aspect an adaptive system should be able to capture to instantiate appropriate adaptation to the user experience. We argue that there is a subtle difference between adaptive user interfaces and adaptive user experiences.

show abstract

Sensing Affective States Using Facial Expression Analysis

Cited by 10 publications

References 28 publications

Automated Analysis of Facial Cues from Videos as a Potential Method for Differentiating Stress and Boredom of Players in Games

Automated Analysis of Facial Cues from Videos as a Potential Method for Differentiating Stress and Boredom of Players in Games

Affective state detection via facial expression analysis within a human–computer interaction context

Adaptive User Experience Based on Detecting User Perplexity

Contact Info

Product

Resources

About