Discovering gender differences in facial emotion recognition via implicit behavioral cues

Abstract

We examine the utility of implicit behavioral cues in the form of EEG brain signals and eye movements for gender recognition (GR) and emotion recognition (ER). Specifically, the examined cues are acquired via low-cost, off-the-shelf sensors. We asked 28 viewers (14 female) to recognize emotions from unoccluded (no mask) as well as partially occluded (eye and mouth masked) emotive faces. Obtained experimental results reveal that (a) reliable GR and ER is achievable with EEG and eye features, (b) differential cognitive processing especially for negative emotions is observed for males and females and (c) some of these cognitive differences manifest under partial face occlusion, as typified by the eye and mouth mask conditions.

Gender and emotion recognition with implicit user signals

“…Conspicuous facial cues are studied in [18] to detect multimedia highlights, while implicit physiological measurements are employed to model emotions induced by music and movie scenes respectively in [20] and [3]. EEG and eye movements are two popular implicit modalities employed for ER, and many works have predicted a ect with a combination of both [7,30,58], or exclusively using either signal [3,19,27,40,46,47,59].…”

“…Conspicuous facial cues are studied to detect multimedia highlights in [9], while physiological measurements are utilized to model emotions induced by music and movie scenes in [29], [30]. EEG and eye movements are two popular modalities employed for ER, and many works have used a combination of both [20], [27], [28] or either signal exclusively [21], [24], [29], [31], [36], [41], [42].…”

“…Building on preliminary results [20], [26], our work makes the following research contributions: (a) While prior works have identified gender differences in emotional behavior, this is one of the first works to expressly perform GR and ER from implicit behavioral signals; (b) Apart from recognition experiments, we show that the employed devices capture meaningful perceptual information, as typified by gender and emotion-specific event related potentials (ERPs), and the extent of fixation over the eyes; (c) We demonstrate a significant performance improvement in GR with deep learning and boosting methods over traditional approaches presented in [20], [26]; (d) The use of minimally intrusive, off-the-shelf and low-cost devices affirms the ecological validity of our study, and the utility of our experimental design for large-scale user profiling.…”

“…We set out to discover gender differences in human visual perception by designing a facial emotion recognition (FER) experiment. Males and females respond differently to affective information [7], [18], [19], [20], and user eye movements are also characteristic of the perceived facial emotion [21], [22], [23], [24], [25], enabling stimulus ER. Our study performed with 28 viewers (14 males) confirms that women achieve superior FER, mirroring prior findings.…”

10 years of EPOC: A scoping review of Emotiv’s portable EEG device

“…By capitalising on the P300, a computer can detect when a target letter flashes on a screen thereby allowing selection of letters without physical interaction. Though therewere several EPOC studies in this review that investigated traditional P300 speller BCI interfaces[34][35][36][37], others harnessed P300 for such purposes as interacting with navigation systems[38] and robotic devices[39]. Suhas, Dhal (40) investigated using ERPs to control a light bulb and a fan, with an eye towards giving physically disabled individuals control of 'smart' appliances.…”