Emotion Recognition in Human–Robot Interaction Using the NAO Robot

Valagkouti, Iro Athina; Troussas, Christos; Krouska, Akrivi; Feidakis, Michalis; Sgouropoulou, Cleo

doi:10.3390/computers11050072

Cited by 18 publications

(5 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus, in these robots, additional materials, such as real or cartoon-like images were used to teach facial expressions by giving explanations, instructions, or feedback to children. In addition, there were studies that focused on detecting the emotional and behavioral states of children with ASD for better affective interaction with ASD children ( Rudovic et al., 2018 ; Kim et al., 2021 ; Silva et al., 2021 ; Lytridis et al., 2022 ; Valagkouti et al., 2022 ). In these studies, different features, such as facial expressions, gestures, and eye gaze duration, were used to measure the engagement and affective state of ASD children.…”

Section: Related Work On Emotion Recognition and Expression Using Robotsmentioning

confidence: 99%

Fully robotic social environment for teaching and practicing affective interaction: Case of teaching emotion recognition skills to children with autism spectrum disorder, a pilot study

et al. 2023

View full text Add to dashboard Cite

21st century brought along a considerable decrease in social interactions, due to the newly emerged lifestyle around the world, which became more noticeable recently of the COVID-19 pandemic. On the other hand, children with autism spectrum disorder have further complications regarding their social interactions with other humans. In this paper, a fully Robotic Social Environment (RSE), designed to simulate the needed social environment for children, especially those with autism is described. An RSE can be used to simulate many social situations, such as affective interpersonal interactions, in which observational learning can take place. In order to investigate the effectiveness of the proposed RSE, it has been tested on a group of children with autism, who had difficulties in emotion recognition, which in turn, can influence social interaction. An A-B-A single case study was designed to show how RSE can help children with autism recognize four basic facial expressions, i.e., happiness, sadness, anger, and fear, through observing the social interactions of two robots speaking about these facial expressions. The results showed that the emotion recognition skills of the participating children were improved. Furthermore, the results showed that the children could maintain and generalize their emotion recognition skills after the intervention period. In conclusion, the study shows that the proposed RSE, along with other rehabilitation methods, can be effective in improving the emotion recognition skills of children with autism and preparing them to enter human social environments.

show abstract

Section: Related Work On Emotion Recognition and Expression Using Robotsmentioning

confidence: 99%

Fully robotic social environment for teaching and practicing affective interaction: Case of teaching emotion recognition skills to children with autism spectrum disorder, a pilot study

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Wearable inertial measurement unit (IMU) sensors, which are made up of accelerometers and gyroscopes, have developed quickly in recent years, making it simple to detect and track human movement. This has made it possible for HAR to be used in a variety of industries, including healthcare, biometrics, and also other fields such as human emotion recognition [ 1 ]. The significance of HAR based on wearable sensors is made clear by the fact that it is not only restricted to exercise-related activities but can also be used for classifying and documenting various daily behaviors such as eating, drinking, brushing teeth, and detecting sleep irregularities.…”

Section: Introductionmentioning

confidence: 99%

Human Activity Recognition Using Attention-Mechanism-Based Deep Learning Feature Combination

Akter,

Ansary,

Khan

et al. 2023

Sensors

View full text Add to dashboard Cite

Human activity recognition (HAR) performs a vital function in various fields, including healthcare, rehabilitation, elder care, and monitoring. Researchers are using mobile sensor data (i.e., accelerometer, gyroscope) by adapting various machine learning (ML) or deep learning (DL) networks. The advent of DL has enabled automatic high-level feature extraction, which has been effectively leveraged to optimize the performance of HAR systems. In addition, the application of deep-learning techniques has demonstrated success in sensor-based HAR across diverse domains. In this study, a novel methodology for HAR was introduced, which utilizes convolutional neural networks (CNNs). The proposed approach combines features from multiple convolutional stages to generate a more comprehensive feature representation, and an attention mechanism was incorporated to extract more refined features, further enhancing the accuracy of the model. The novelty of this study lies in the integration of feature combinations from multiple stages as well as in proposing a generalized model structure with CBAM modules. This leads to a more informative and effective feature extraction technique by feeding the model with more information in every block operation. This research used spectrograms of the raw signals instead of extracting hand-crafted features through intricate signal processing techniques. The developed model has been assessed on three datasets, including KU-HAR, UCI-HAR, and WISDM datasets. The experimental findings showed that the classification accuracies of the suggested technique on the KU-HAR, UCI-HAR, and WISDM datasets were 96.86%, 93.48%, and 93.89%, respectively. The other evaluation criteria also demonstrate that the proposed methodology is comprehensive and competent compared to previous works.

show abstract

“…A systematic review by Stock-Homburg (2022) offered insights into humans' recognition and responses to artificial emotions in HRI [11]. Valagkouti (2022) focused on affective computing and game-based environmental awareness using NAO robots, demonstrating real-world applications [12]. These studies collectively contribute to the advancement of HRI through GER, addressing challenges and improving the naturalness of HRIs.…”

Section: Introductionmentioning

confidence: 99%

Enhancing Human-Robot Interaction Through Group Emotion Recognition

Mousa,

Mahdi Nasser

2023

ijci

View full text Add to dashboard Cite

- This article explores within the field of Human-Robot Interaction (HRI), focusing on the complicated relationship between emotions, decision-making, and robot behaviors. Emotions are essential to effective communication and interaction, requiring the development of emotion recognition systems in robots. The article explores both individual and group emotion recognition, including microexpressions and macroexpressions. Group emotion dynamics, encompassing phenomena like emotional contagion, convergence, and social influence, are separated to understand how emotions combine within collective settings. A concept, Group Emotion Recognition (GER), is introduced, providing a framework for recognizing emotions within groups. GER involves proximity metrics, emotion classification, and entropy-based analysis to quantify emotion diversity. The article also outlines how GER can enhance user engagement, personalize interactions, improve group dynamics, and foster social acceptance in various human-robot interaction scenarios. Decision-making based on GER, driven by positive or negative emotion labels, is discussed, highlighting the adaptability and sensitivity required for effective human-robot interactions. Ethical considerations regarding the use of emotion recognition technology are addressed throughout the article, emphasizing responsible implementation. Overall, this work lays a solid foundation for advancing the field of HRI by integrating emotion recognition and decision-making to create emotionally intelligent and socially aware robots.

show abstract

Emotion Recognition in Human–Robot Interaction Using the NAO Robot

Cited by 18 publications

References 31 publications

Fully robotic social environment for teaching and practicing affective interaction: Case of teaching emotion recognition skills to children with autism spectrum disorder, a pilot study

Fully robotic social environment for teaching and practicing affective interaction: Case of teaching emotion recognition skills to children with autism spectrum disorder, a pilot study

Human Activity Recognition Using Attention-Mechanism-Based Deep Learning Feature Combination

Enhancing Human-Robot Interaction Through Group Emotion Recognition

Contact Info

Product

Resources

About