How are you feeling? Multimodal Emotion Learning for Socially-Assistive Robot Navigation

Bera, Aniket; Randhavane, Tanmay; Prinja, Rohan; Kapsaskis, Kyra; Wang, Austin; Manocha, Dinesh

doi:10.1109/fg47880.2020.00141

Cited by 10 publications

(6 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An adjacent topic is the integration of emotional cues into Human Robot Interaction [322] and potentially robot navigation. Narayanan et al [323] inferred overall mood from gaits, which were used to adapt the robot´s path and Bera et al [324] used trajectories and faces, as did Gines et al [325]. Yelwande et al [326] compared different path planning strategies that utilise information about the human´s emotional state.…”

Section: Discussionmentioning

confidence: 99%

A Survey on Human-aware Robot Navigation

Möller¹,

Furnari²,

Battiato³

et al. 2021

Preprint

View full text Add to dashboard Cite

Intelligent systems are increasingly part of our everyday lives and have been integrated seamlessly to the point where it is difficult to imagine a world without them. Physical manifestations of those systems on the other hand, in the form of embodied agents or robots, have so far been used only for specific applications and are often limited to functional roles (e.g. in the industry, entertainment and military fields). Given the current growth and innovation in the research communities concerned with the topics of robot navigation, human-robot-interaction and human activity recognition, it seems like this might soon change. Robots are increasingly easy to obtain and use and the acceptance of them in general is growing. However, the design of a socially compliant robot that can function as a companion needs to take various areas of research into account. This paper is concerned with the navigation aspect of a socially-compliant robot and provides a survey of existing solutions for the relevant areas of research as well as an outlook on possible future directions.

show abstract

Section: Discussionmentioning

confidence: 99%

A Survey on Human-aware Robot Navigation

Möller¹,

Furnari²,

Battiato³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Facial expression recognition has been well studied in human-robot interaction (HRI) (e.g., Cid et al, 2014 ; Meghdari et al, 2016 ; Simul et al, 2016 ; Bera et al, 2019 ). As deep learning methods have become popular, facial expression recognition nowadays mostly consists of preprocessing the facial images and directly feeding them into deep networks to predict an output ( Li and Deng, 2020 ).…”

Section: Related Workmentioning

confidence: 99%

ExGenNet: Learning to Generate Robotic Facial Expression Using Facial Expression Recognition

et al. 2022

View full text Add to dashboard Cite

The ability of a robot to generate appropriate facial expressions is a key aspect of perceived sociability in human-robot interaction. Yet many existing approaches rely on the use of a set of fixed, preprogrammed joint configurations for expression generation. Automating this process provides potential advantages to scale better to different robot types and various expressions. To this end, we introduce ExGenNet, a novel deep generative approach for facial expressions on humanoid robots. ExGenNets connect a generator network to reconstruct simplified facial images from robot joint configurations with a classifier network for state-of-the-art facial expression recognition. The robots’ joint configurations are optimized for various expressions by backpropagating the loss between the predicted expression and intended expression through the classification network and the generator network. To improve the transfer between human training images and images of different robots, we propose to use extracted features in the classifier as well as in the generator network. Unlike most studies on facial expression generation, ExGenNets can produce multiple configurations for each facial expression and be transferred between robots. Experimental evaluations on two robots with highly human-like faces, Alfie (Furhat Robot) and the android robot Elenoide, show that ExGenNet can successfully generate sets of joint configurations for predefined facial expressions on both robots. This ability of ExGenNet to generate realistic facial expressions was further validated in a pilot study where the majority of human subjects could accurately recognize most of the generated facial expressions on both the robots.

show abstract

“…Similarly, Bera et al [13] use different data inputs captured from different perspectives, from the robot's camera and a surveillance camera. They propose a system for socially-aware navigation to analyze the emotional behavior from people's faces and trajectories.…”

Section: Related Workmentioning

confidence: 99%

Multimodal Emotional Understanding in Robotics

Heredia

Cardinale

Dongo

et al. 2022

Ambient Intelligence and Smart Environments

View full text Add to dashboard Cite

In the context of Human-Robot Interaction (HRI), emotional understanding is becoming more popular because it turns robots more humanized and user-friendly. Giving a robot the ability to recognize emotions has several difficulties due to the limits of the robots’ hardware and the real-world environments in which it works. In this sense, an out-of-robot approach and a multimodal approach can be the solution. This paper presents the implementation of a previous proposed multi-modal emotional system in the context of social robotics; that works on a server and bases its prediction in four modalities as inputs (face, posture, body, and context features) captured through the robot’s sensors; the predicted emotion triggers some robot behavior changes. Working on a server allows overcoming the robot’s hardware limitations but gaining some delay in the communication. Working with several modalities allows facing complex real-world scenarios strongly and adaptively. This research is focused on analyzing, explaining, and arguing the usability and viability of an out-of-robot and multimodal approach for emotional robots. Functionality tests were applied with the expected results, demonstrating that the entire proposed system takes around two seconds; delay justified on the deep learning models used, which are improvable. Regarding the HRI evaluations, a brief discussion about the remaining assessments is presented, explaining how difficult it can be a well-done evaluation of this work. The demonstration of the system functionality can be seen at https://youtu.be/MYYfazSa2N0.

show abstract

How are you feeling? Multimodal Emotion Learning for Socially-Assistive Robot Navigation

Cited by 10 publications

References 48 publications

A Survey on Human-aware Robot Navigation

A Survey on Human-aware Robot Navigation

ExGenNet: Learning to Generate Robotic Facial Expression Using Facial Expression Recognition

Multimodal Emotional Understanding in Robotics

Contact Info

Product

Resources

About