Emotion analysis from faces for social robotics

Greco, Antonio; Roberto, Antonio; Saggese, Alessia; Vento, Mario; Vigilante, Vincenzo

doi:10.1109/smc.2019.8914039

Cited by 14 publications

(10 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, it makes the treatment of multimodal data more comfortable because of the gain of resources for processing. Greco et al [8], for example, add a device (Nvidia Jetson TX2) with a graphics card into the Pepper robot that performs face detection and emotion recognition, and the robot modifies its voice and movements. The architectures tested are MobileNet and ResNet-18 with a temporal module; in the end, their better results are achieved by a ResNet-18 plus a temporal average module.…”

Section: Related Workmentioning

confidence: 99%

“…They argue that the robot can perform a good emotion recognition process in real-time by adding an extra device. Unlike Greco et al [8], Martinez et al [9] use an external server instead of the extra device. The models used might be more complex, and their updating or deployment would require more effort.…”

Section: Related Workmentioning

confidence: 99%

“…Besides, developing the emotion system outside leaves more resources for fundamental robot applications, such as mapping, navigation, and routing tasks. With hybrid approaches, as presented in [8], the addition of extra devices inside the robot does not have the flexibility in the deployment environment.…”

Section: Multimodal Emotion Recognition Systemmentioning

confidence: 99%

See 2 more Smart Citations

Multimodal Emotional Understanding in Robotics

Heredia

Cardinale

Dongo

et al. 2022

Ambient Intelligence and Smart Environments

View full text Add to dashboard Cite

In the context of Human-Robot Interaction (HRI), emotional understanding is becoming more popular because it turns robots more humanized and user-friendly. Giving a robot the ability to recognize emotions has several difficulties due to the limits of the robots’ hardware and the real-world environments in which it works. In this sense, an out-of-robot approach and a multimodal approach can be the solution. This paper presents the implementation of a previous proposed multi-modal emotional system in the context of social robotics; that works on a server and bases its prediction in four modalities as inputs (face, posture, body, and context features) captured through the robot’s sensors; the predicted emotion triggers some robot behavior changes. Working on a server allows overcoming the robot’s hardware limitations but gaining some delay in the communication. Working with several modalities allows facing complex real-world scenarios strongly and adaptively. This research is focused on analyzing, explaining, and arguing the usability and viability of an out-of-robot and multimodal approach for emotional robots. Functionality tests were applied with the expected results, demonstrating that the entire proposed system takes around two seconds; delay justified on the deep learning models used, which are improvable. Regarding the HRI evaluations, a brief discussion about the remaining assessments is presented, explaining how difficult it can be a well-done evaluation of this work. The demonstration of the system functionality can be seen at https://youtu.be/MYYfazSa2N0.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Multimodal Emotional Understanding in Robotics

Heredia

Cardinale

Dongo

et al. 2022

Ambient Intelligence and Smart Environments

View full text Add to dashboard Cite

show abstract

“…For their foundation, these studies rely on contributions from robotics (e.g., [1,11,156]) and HRI [121,122,169,225]. Further studies are rooted in human-computer interaction (e.g., [3,4,21,99,140,173], engineering [171], and philosophy [101].…”

Section: Framework Of the Overviewmentioning

confidence: 99%

Survey of Emotions in Human–Robot Interactions: Perspectives from Robotic Psychology on 20 Years of Research

Stock‐Homburg

2021

Int J of Soc Robotics

View full text Add to dashboard Cite

Knowledge production within the interdisciplinary field of human–robot interaction (HRI) with social robots has accelerated, despite the continued fragmentation of the research domain. Together, these features make it hard to remain at the forefront of research or assess the collective evidence pertaining to specific areas, such as the role of emotions in HRI. This systematic review of state-of-the-art research into humans’ recognition and responses to artificial emotions of social robots during HRI encompasses the years 2000–2020. In accordance with a stimulus–organism–response framework, the review advances robotic psychology by revealing current knowledge about (1) the generation of artificial robotic emotions (stimulus), (2) human recognition of robotic artificial emotions (organism), and (3) human responses to robotic emotions (response), as well as (4) other contingencies that affect emotions as moderators.

show abstract

“…Real-time data acquisition and analysis in conjunction with the information already gathered, results in better interpretation of the data [1], which is clearly visible in inter alia numerous Internet of Things (IoT)-based healthcare applications [6] or in robotics [7], or in autonomous systems [8]. The use of a humanoid robot gives additional benefits in building the company's relationship with a client [4,7,[9][10][11].The main aim of this work was to extend humanoid robots, which could be considered as both social and service robots using various smart sensors. Another innovation in this work is the HISS system, its idea, design and prototype implementation, fully developed by the authors of this work.…”

mentioning

confidence: 99%

Human Interaction Smart Subsystem—Extending Speech-Based Human-Robot Interaction Systems with an Implementation of External Smart Sensors

Podpora

Gardecki

Beniak

et al. 2020

Sensors

View full text Add to dashboard Cite

This paper presents a more detailed concept of Human-Robot Interaction systems architecture. One of the main differences between the proposed architecture and other ones is the methodology of information acquisition regarding the robot's interlocutor. In order to obtain as much information as possible before the actual interaction took place, a custom Internet-of-Things-based sensor subsystems connected to Smart Infrastructure was designed and implemented, in order to support the interlocutor identification and acquisition of initial interaction parameters. The Artificial Intelligence interaction framework of the developed robotic system (including humanoid Pepper with its sensors and actuators, additional local, remote and cloud computing services) is being extended with the use of custom external subsystems for additional knowledge acquisition: device-based human identification, visual identification and audio-based interlocutor localization subsystems. These subsystems were deeply introduced and evaluated in this paper, presenting the benefits of integrating them into the robotic interaction system. In this paper a more detailed analysis of one of the external subsystems-Bluetooth Human Identification Smart Subsystem-was also included. The idea, use case, and a prototype, integration of elements of Smart Infrastructure systems and the prototype implementation were performed in a small front office of the Weegree company as a decent test-bed application area.The days, when simply having a robot was impressive, are gone. The technology should be more than just presenting content. It should be truly interactive. The authors of this work evaluated methods for the purpose of measurement and influencing the Human-Robot interaction User Experience, (see Reference [1]), concluding that the robot's interaction is perceived as more natural if the robot is able to personalize its communication content towards a particular interlocutor. It became clear, that the system should be able to get as much information as possible during and before the conversation takes place [2]. One of the key aspects in a healthy conversation is emotional state recognition. All the improvements of the robots presented in this paper are aiming for natural, real integration with human customers with the implementation of Artificial Intelligence [3,4].The authors of this work have indicated extending the robot's capabilities by adding additional external wireless sensors in Reference [2], however without Smart Infrastructure. Implementation of a remote PID (Passive Infrared Detector) sensor in order to get a signal "someone is coming" seemed to be enough at that stage.In this paper an extended approach is being presented, introducing Human Identification Smart Subsystems (HISS)-a set of external sensors, which enable us to obtain much more valuable information, resulting from the communication with the Smart Infrastructure [5] or databases of the customer. The idea presented in this paper has been successfully verified and implemented and the con...

show abstract

Emotion analysis from faces for social robotics

Cited by 14 publications

References 29 publications

Multimodal Emotional Understanding in Robotics

Multimodal Emotional Understanding in Robotics

Survey of Emotions in Human–Robot Interactions: Perspectives from Robotic Psychology on 20 Years of Research

Human Interaction Smart Subsystem—Extending Speech-Based Human-Robot Interaction Systems with an Implementation of External Smart Sensors

Contact Info

Product

Resources

About