Guidelines for Collecting Automatic Facial Expression Detection Data Synchronized with a Dynamic Stimulus in Remote Moderated User Tests

Giroux, Félix; Léger, Pierre‐Majorique; Brieugne, David; Courtemanche, François; Bouvier, Frédérique; Chen, Shang-Lin; Tazi, Salima; Rucco, Emma; Frédette, Marc; Coursaris, Constantinos K.; Sénécal, Sylvain

doi:10.1007/978-3-030-78462-1_18

Cited by 13 publications

(3 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sometimes users do not feel comfortable talking about their experiences in a usability study, such as letting people know what they really think or feel about a particular interface. This might be due to the fact that they feel, it is socially inappropriate or they feel that they are the problem rather than the interface, this has been noted in older participants (Gross et al, 1997;Giroux et al, 2021). Objective measures do not rely on a users' experience or assessment, Emotion detection on webpages rather they record and measure time and task completion (Bergstrom and Schall, 2014) as user attributes, one novel approach we applied was adding physiological attributes.…”

Section: Literature Reviewmentioning

confidence: 99%

Emotion detection on webpages using biosensors integrated to a window-based dynamic control system

Isiaka

Abdulkarim²,

Mwitondi

et al. 2021

IJICC

View full text Add to dashboard Cite

PurposeDetecting emotion on user experience of web applications and browsing is important in many ways. Web designers and developers find such approach quite useful in enhancing navigational features of webpages, and biomedical personnel regularly use computer simulations to monitor and control the behaviour of patients. On the other hand, law enforcement agents rely on human physiological functions to determine the likelihood of falsehood in interrogations. Quite often, online user experience is studied via tangible measures such as task completion time, surveys and comprehensive tests from which data attributes are generated. Prediction of users' emotion and behaviour in some of these cases depends mostly on task completion time and number of clicks per given time interval. However, such approaches are generally subjective and rely heavily on distributional assumptions making the results prone to recording errors.Design/methodology/approachThe authors propose a novel method-a window dynamic control system that addresses the foregoing issues. Primary data were obtained from laboratory experiments during which forty-four volunteers had their synchronised physiological readings, skin conductance response (SCR), skin temperature (ST), eye movement behaviour and users’ activity attributes taken using biosensors. The window-based dynamic control system (PHYCOB I) is integrated to the biosensor which collects secondary data attributes from these synchronised physiological readings and uses them for two purposes. For both detection of optimal emotional responses and users' stress levels. The method's novelty derives from its ability to integrate physiological readings and eye movement records to identify hidden correlates on a webpage.FindingsResults show that the control system detects basic emotions and outperforms other conventional models in terms of both accuracy and reliability, when subjected to model comparison that is, the average recoverable natural structures for the three models with respect to accuracy and reliability are more consistent within the window-based control system environment than with the conventional methods.Research limitations/implicationsThe paper is limited to using a window control system to detect emotions on webpages, while integrated to biosensors and eye-tracker.Originality/valueThe originality of the proposed model is its resistance to overfitting and its ability to automatically assess human emotion (stress levels) while dealing with specific web contents. The latter is particularly important in that it can be used to predict which contents of webpages cause stress-induced emotions to users when involved in online activities.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Emotion detection on webpages using biosensors integrated to a window-based dynamic control system

Isiaka

Abdulkarim²,

Mwitondi

et al. 2021

IJICC

View full text Add to dashboard Cite

show abstract

“…Considering the nature of the COVID-19 pandemic, a remote experimental laboratory was made mandatory. The experiment followed guidelines established for remote data collection [78,79].…”

Section: Experimental Designmentioning

confidence: 99%

Comparing the Effectiveness of Speech and Physiological Features in Explaining Emotional Responses during Voice User Interface Interactions

et al. 2022

View full text Add to dashboard Cite

The rapid rise of voice user interface technology has changed the way users traditionally interact with interfaces, as tasks requiring gestural or visual attention are swapped by vocal commands. This shift has equally affected designers, required to disregard common digital interface guidelines in order to adapt to non-visual user interaction (No-UI) methods. The guidelines regarding voice user interface evaluation are far from the maturity of those surrounding digital interface evaluation, resulting in a lack of consensus and clarity. Thus, we sought to contribute to the emerging literature regarding voice user interface evaluation and, consequently, assist user experience professionals in their quest to create optimal vocal experiences. To do so, we compared the effectiveness of physiological features (e.g., phasic electrodermal activity amplitude) and speech features (e.g., spectral slope amplitude) to predict the intensity of users’ emotional responses during voice user interface interactions. We performed a within-subjects experiment in which the speech, facial expression, and electrodermal activity responses of 16 participants were recorded during voice user interface interactions that were purposely designed to elicit frustration and shock, resulting in 188 analyzed interactions. Our results suggest that the physiological measure of facial expression and its extracted feature, automatic facial expression-based valence, is most informative of emotional events lived through voice user interface interactions. By comparing the unique effectiveness of each feature, theoretical and practical contributions may be noted, as the results contribute to voice user interface literature while providing key insights favoring efficient voice user interface evaluation.

show abstract

“…A large number of automatic facial expression analysis solutions have been proposed in recent years [6,7,8]. However, several works have shown that models trained on adult expression datasets do not generalize well on child facial expression recognition tasks [4] as children and adults vary significantly in how they display expressions of emotions, leading to different manifestations in facial expressions [9].…”

Section: Introductionmentioning

confidence: 99%

Statistical, Spectral and Graph Representations for Video-Based Facial Expression Recognition in Children

Abbasi

Song

Güneş

2022

ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Child facial expression recognition is a relatively less investigated area within affective computing. Children's facial expressions differ significantly from adults; thus, it is necessary to develop emotion recognition frameworks that are more objective, descriptive and specific to this target user group. In this paper we propose the first approach that (i) constructs video-level heterogeneous graph representation for facial expression recognition in children, and (ii) predicts children's facial expressions using the automatically detected Action Units (AUs). To this aim, we construct three separate length-independent representations, namely, statistical, spectral and graph at video-level for detailed multi-level facial behaviour decoding (AU activation status, AU temporal dynamics and spatio-temporal AU activation patterns, respectively). Our experimental results on the LIRIS Children Spontaneous Facial Expression Video Database demonstrate that combining these three feature representations provides the highest accuracy for expression recognition in children.

show abstract

Guidelines for Collecting Automatic Facial Expression Detection Data Synchronized with a Dynamic Stimulus in Remote Moderated User Tests

Cited by 13 publications

References 22 publications

Emotion detection on webpages using biosensors integrated to a window-based dynamic control system

Emotion detection on webpages using biosensors integrated to a window-based dynamic control system

Comparing the Effectiveness of Speech and Physiological Features in Explaining Emotional Responses during Voice User Interface Interactions

Statistical, Spectral and Graph Representations for Video-Based Facial Expression Recognition in Children

Contact Info

Product

Resources

About