Improving Human–Robot Interaction by Enhancing NAO Robot Awareness of Human Facial Expression

Abstract

An intriguing challenge in the human–robot interaction field is the prospect of endowing robots with emotional intelligence to make the interaction more genuine, intuitive, and natural. A crucial aspect in achieving this goal is the robot’s capability to infer and interpret human emotions. Thanks to its design and open programming platform, the NAO humanoid robot is one of the most widely used agents for human interaction. As with person-to-person communication, facial expressions are the privileged channel for recognizing the interlocutor’s emotional expressions. Although NAO is equipped with a facial expression recognition module, specific use cases may require additional features and affective computing capabilities that are not currently available. This study proposes a highly accurate convolutional-neural-network-based facial expression recognition model that is able to further enhance the NAO robot’ awareness of human facial expressions and provide the robot with an interlocutor’s arousal level detection capability. Indeed, the model tested during human–robot interactions was 91% and 90% accurate in recognizing happy and sad facial expressions, respectively; 75% accurate in recognizing surprised and scared expressions; and less accurate in recognizing neutral and angry expressions. Finally, the model was successfully integrated into the NAO SDK, thus allowing for high-performing facial expression classification with an inference time of 0.34 ± 0.04 s.

Automated Affective Computing Based on Bio-Signals Analysis and Deep Learning Approach

“…In addition, respiration (RSP) patterns provide important informat Furthermore, up to now, the emotion recognition methods are classified into trait-based (i.e., facial expression, gestures, and speech) and physiological-state-based approaches [31]. Although the former have been majorly explored and used for emotion classification purposes [32], they may be prone to subconscious control, resulting in inaccurate performance. On the contrary, approaches based on physiological signals (which cannot be voluntarily altered) can reflect people's emotions more accurately than non-physiological ones [33] and are growing increasingly prominent.…”

Effect of robot’s vertical body movement on its perceived emotion: A preliminary study on vertical oscillation and transition

“…Finally, conducting real-world human–robot interaction experiments with actual constraints would be an important next step for this study. For a real-word interaction with human, ability of the robot to adapt its behavior based on human’s reaction to prevent the probable short fall in the interaction is one of the essential issues, e.g., producing empathetic response to the emotion of the interlocutor [ 59 ]. Research on the human-like interaction skills for the robots to improve emotion recognition and consequently produce adequate response as well as emotional expression is required as another important future work.…”

Empowering Smart Aging: Insights into the Technical Architecture of the e-VITA Virtual Coaching System for Older Adults

“…The coaching devices integrated into the e-VITA platform are presented in Figure 3 and include the following: NAO, a compact humanoid robot, is widely employed in studies centered on human–robot interaction [ 35 , 36 , 37 ]. The NAO robot serves as a versatile platform for diverse interactive tasks, using speech, gestures, and other interactive behaviors to engage users.…”

“…1. NAO, a compact humanoid robot, is widely employed in studies centered on humanrobot interaction [35][36][37]. The NAO robot serves as a versatile platform for diverse HRV (ms), HR (bpm), respiratory rate (rpm), burned calories, inactivity time (h), steps, sleep timing (h).…”