Classifying human emotions in HRI: applying global optimization model to EEG brain signals

Abstract

Significant efforts have been made in the past decade to humanize both the form and function of social robots to increase their acceptance among humans. To this end, social robots have recently been combined with brain-computer interface (BCI) systems in an attempt to give them an understanding of human mental states, particularly emotions. However, emotion recognition using BCIs poses several challenges, such as subjectivity of emotions, contextual dependency, and a lack of reliable neuro-metrics for real-time processing of emotions. Furthermore, the use of BCI systems introduces its own set of limitations, such as the bias-variance trade-off, dimensionality, and noise in the input data space. In this study, we sought to address some of these challenges by detecting human emotional states from EEG brain activity during human-robot interaction (HRI). EEG signals were collected from 10 participants who interacted with a Pepper robot that demonstrated either a positive or negative personality. Using emotion valence and arousal measures derived from frontal brain asymmetry (FBA), several machine learning models were trained to classify human's mental states in response to the robot personality. To improve classification accuracy, all proposed classifiers were subjected to a Global Optimization Model (GOM) based on feature selection and hyperparameter optimization techniques. The results showed that it is possible to classify a user's emotional responses to the robot's behavior from the EEG signals with an accuracy of up to 92%. The outcome of the current study contributes to the first level of the Theory of Mind (ToM) in Human-Robot Interaction, enabling robots to comprehend users' emotional responses and attribute mental states to them. Our work advances the field of social and assistive robotics by paving the way for the development of more empathetic and responsive HRI in the future.

Robots as Mental Health Coaches: A Study of Emotional Responses to Technology-Assisted Stress Management Tasks Using Physiological Signals

“…To address the limitations in the past literature concerning uncontrolled experimental design and lack of objective measurements, we compared two groups of participants who engaged in a deep breathing exercise either mediated by a Pepper robot or by a laptop. Additionally, we evaluated the effectiveness of each technology in reducing stress by collecting physiological signals such as GSRs and EEG brain activity, which is a novel approach in HRI research for objective measurement of the users' experience [10,[17][18][19].…”

“…On the other hand, emotional valence can be evaluated by measuring changes in EEG brain activity using the well-known neurometric of Frontal Alpha Asymmetry (FAA) [18,[31][32][33]. FAA refers to the lateralization (change between the right and left hemispheres) of the alpha-band brain activity in the frontal brain region.…”

“…To counteract this issue, the field of HRI can borrow inspiration from affective computing research [18][19][20], in which various physiological signals such as heart rate, skin conductance, respiration rate, and brain activity are used as objective measures of stress levels and emotional responses [20][21][22]. Unlike self-reports that can only be collected once the interaction with the robot is over, physiological signals are time series data that can be recorded with high temporal resolution during the interaction [19].…”

“…Affective computing refers to an interdisciplinary field of research that aims to develop systems that can recognize and respond to human emotions [23]. The theories and methods emerging from this field have tremendous potential to improve the quality of HRI by endowing the robots with the ability to recognize human emotions and appropriately respond to them [18,20,23]. This is especially important in application domains such as healthcare, in which understanding a user's emotional states can help create personalized and more effective HRI [24].…”