Recognition of Emotions in Video Clips: The Self-Assessment Manikin Validation

Handayani, Dini; Wahab, Abdul; Yaacob, Hamwira Sakti

doi:10.12928/telkomnika.v13i4.2735

Cited by 16 publications

(14 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Compared with colorless Circumplex (Figure 2(a)), Healey et al [43] showed that colored Circumplex was preferred by users. Following this, we selected four colors based on a simplified version of Itten's color system [49,98], which has been shown to be intuitive and easy for users to understand [13,40].…”

Section: Designing Our Mobile Annotation Methodsmentioning

confidence: 99%

RCEA: Real-time, Continuous Emotion Annotation for Collecting Precise Mobile Video Ground Truth Labels

Zhang

Ali

Wang³

et al. 2020

Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

Collecting accurate and precise emotion ground truth labels for mobile video watching is essential for ensuring meaningful predictions. However, video-based emotion annotation techniques either rely on post-stimulus discrete self-reports, or allow real-time, continuous emotion annotations (RCEA) only for desktop settings. Following a user-centric approach, we designed an RCEA technique for mobile video watching, and validated its usability and reliability in a controlled, indoor (N=12) and later outdoor (N=20) study. Drawing on physiological measures, interaction logs, and subjective workload reports, we show that (1) RCEA is perceived to be usable for annotating emotions while mobile video watching, without increasing users' mental workload (2) the resulting time-variant annotations are comparable with intended emotion attributes of the video stimuli (classification error for valence: 8.3%; arousal: 25%). We contribute a validated annotation technique and associated annotation fusion method, that is suitable for collecting fine-grained emotion annotations while users watch mobile videos.

show abstract

Section: Designing Our Mobile Annotation Methodsmentioning

confidence: 99%

RCEA: Real-time, Continuous Emotion Annotation for Collecting Precise Mobile Video Ground Truth Labels

Zhang

Ali

Wang³

et al. 2020

Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

show abstract

“…The SAM questionnaire has been used in health research (Jayanti and Whipple, 2008) and has also been used to rate the affective dimensions of valence and arousal of individuals while watching videos (Setz et al, 2009; Handayani et al, 2015).…”

Section: Methodsmentioning

confidence: 99%

The Cognitive Dimension and the Affective Dimension in the Patient’s Experience

2019

View full text Add to dashboard Cite

This article deals with the experience of the specific client of health services, that is, the patient. Satisfaction questionnaires are usually applied to assess patient experience. However, this tool provides only a cognitive evaluation; it does not afford an affective dimension of the experience. The objective of the present study is to verify the relationship between the cognitive dimension of patient experience, collected through questionnaires, and the affective dimension, derived from the analysis of neurophysiological data. We propose a novel methodology that integrates physiological data collected by facial expression analysis to identify patients’ emotions. A first, qualitative procedure was carried out to define the patient journey. This was recorded on video and later used in the experiment. The experiment collected information from the participants using two techniques. First, as they viewed the videos, facial expression analysis (FEA) was applied to assess their responses. Second, after they watched the videos, traditional questionnaires were presented. The results provided by the two techniques were then compared. The results show that there is no relationship between the emotional valence reported by questionnaires and the neurophysiological data. This reflects the two different dimensions of the experience, one cognitive and the other affective. Both facilitate the understanding of patient satisfaction.

show abstract

“…UI components like frame, dot, light, text were considered as well as attributes such as position, size and transparency [14,19]. The color of the component indicates the emotion they were annotating currently [13]. Four colors (HEX values = #eecdac, #7fc087, #879af0, #f4978e for quadrants one to four respectively) provided feedback to users on which emotion they were currently annotating.…”

Section: Visual Annotation Feedbackmentioning

confidence: 99%

“…Four colors (HEX values = #eecdac, #7fc087, #879af0, #f4978e for quadrants one to four respectively) provided feedback to users on which emotion they were currently annotating. We selected four colors based on a simplified version of Itten's color system [34], which has been shown to be intuitive and easy for users to understand [13].…”

Section: Visual Annotation Feedbackmentioning

confidence: 99%

See 1 more Smart Citation

Designing Real-time, Continuous Emotion Annotation Techniques for 360° VR Videos

Xue

Ghosh

Ding

et al. 2020

Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

With the increasing availability of head-mounted displays (HMDs) that show immersive 360 • VR content, it is important to understand to what extent these immersive experiences can evoke emotions. Typically to collect emotion ground truth labels, users rate videos through postexperience self-reports that are discrete in nature. However, post-stimuli self-reports are temporally imprecise, especially after watching 360 • videos. In this work, we design six continuous emotion annotation techniques for the Oculus Rift HMD aimed at minimizing workload and distraction. Based on a co-design session with six experts, we contribute HaloLight and DotSize, two continuous annotation methods deemed unobtrusive and easy to understand. We discuss the next challenges for evaluating the usability of these techniques, and reliability of continuous annotations.

show abstract

Recognition of Emotions in Video Clips: The Self-Assessment Manikin Validation

Abstract: Abstract

Cited by 16 publications

References 13 publications

RCEA: Real-time, Continuous Emotion Annotation for Collecting Precise Mobile Video Ground Truth Labels

RCEA: Real-time, Continuous Emotion Annotation for Collecting Precise Mobile Video Ground Truth Labels

The Cognitive Dimension and the Affective Dimension in the Patient’s Experience

Designing Real-time, Continuous Emotion Annotation Techniques for 360° VR Videos

Contact Info

Product

Resources

About