Adolescents Environmental Emotion Perception by Integrating EEG and Eye Movements

Abstract

Giving a robot the ability to perceive emotion in its environment can improve human-robot interaction (HRI), thereby facilitating more human-like communication. To achieve emotion recognition in different built environments for adolescents, we propose a multi-modal emotion intensity perception method using an integration of electroencephalography (EEG) and eye movement information. Specifically, we first develop a new stimulus video selection method based on computation of normalized arousal and valence scores according to subjective feedback from participants. Then, we establish a valence perception sub-model and an arousal sub-model by collecting and analyzing emotional EEG and eye movement signals, respectively. We employ this dual recognition method to perceive emotional intensities synchronously in two dimensions. In the laboratory environment, the best recognition accuracies of the modality fusion for the arousal and valence dimensions are 72.8 and 69.3%. The experimental results validate the feasibility of the proposed multi-modal emotion recognition method for environment emotion intensity perception. This promising tool not only achieves more accurate emotion perception for HRI systems but also provides an alternative approach to quantitatively assess environmental psychology.

fNIRS-Based Upper Limb Motion Intention Recognition Using an Artificial Neural Network for Transhumeral Amputees

“…Based on the conditions during the signal acquisition process, further steps were taken, such as filtration and channel selection. This makes a difference, as documented by [ 22 , 58 ]. The results show the potential usability of the presented framework in real-time applications and is a step towards enhanced motion prediction in BMI applications.…”

Analysis of Classroom Learning Behaviors Based on Internet of Things and Image Processing

“…When it comes to multimodal feature extraction, the initial approach is to fuse the countenance data with audio signals, trying to improve the reliability and effectiveness of emotion computing. Su et al [28] synthetized the EEG information with eye movement signals, and classified emotions with a multi-modal DL deep neural network (DNN). Kessous et al [29] fused the extracted speech and countenance features, and classified emotions using an SVM.…”