Robot gains social intelligence through multimodal deep reinforcement learning

Qureshi, Ahmed H.; Nakamura, Yutaka; Yoshikawa, Yuichiro; Ishiguro, Hiroshi

doi:10.1109/humanoids.2016.7803357

Cited by 89 publications

(99 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…However, we want to move away from feature engineering and formulate our human-robot interaction scenario as a deep reinforcement learning problem. Recent studies in HRI showed impressive results in employing deep reinforcement learning for various applications [14,15,12]. The main challenge for deep learning approaches is the lack of training data from human studies but we plan to tackle this problem using our current Bayesian-based model to simulate human behaviour data as a prior for the deep reinforcement learning model.…”

Section: Resultsmentioning

confidence: 99%

Exploring Temporal Dependencies in Multimodal Referring Expressions with Mixed Reality

Sibirtseva

Ghadirzadeh

Leite

et al. 2019

Virtual, Augmented and Mixed Reality. Applications and Case Studies

View full text Add to dashboard Cite

In collaborative tasks, people rely both on verbal and nonverbal cues simultaneously to communicate with each other. For humanrobot interaction to run smoothly and naturally, a robot should be equipped with the ability to robustly disambiguate referring expressions. In this work, we propose a model that can disambiguate multimodal fetching requests using modalities such as head movements, hand gestures, and speech. We analysed the acquired data from mixed reality experiments and formulated a hypothesis that modelling temporal dependencies of events in these three modalities increases the model's predictive power. We evaluated our model on a Bayesian framework to interpret referring expressions with and without exploiting the temporal prior.

show abstract

Section: Resultsmentioning

confidence: 99%

Exploring Temporal Dependencies in Multimodal Referring Expressions with Mixed Reality

Sibirtseva

Ghadirzadeh

Leite

et al. 2019

Virtual, Augmented and Mixed Reality. Applications and Case Studies

View full text Add to dashboard Cite

show abstract

“…Naturally, the same trend can be observed regarding the problem of adaptation in HRI. One of the pioneer works was conducted by Qureshi in 2017 [29] where a Deep Q-Network [27] was used to learn a mapping from visual input to one of the several predefined actions for greeting people.…”

Section: Related Workmentioning

confidence: 99%

Fast Adaptation with Meta-Reinforcement Learning for Trust Modelling in Human-Robot Interaction

Gao

Sibirtseva

Castellano

et al. 2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

In socially assistive robotics, an important research area is the development of adaptation techniques and their effect on human-robot interaction. We present a metalearning based policy gradient method for addressing the problem of adaptation in human-robot interaction and also investigate its role as a mechanism for trust modelling. By building an escape room scenario in mixed reality with a robot, we test our hypothesis that bi-directional trust can be influenced by different adaptation algorithms. We found that our proposed model increased the perceived trustworthiness of the robot and influenced the dynamics of gaining human's trust. Additionally, participants evaluated that the robot perceived them as more trustworthy during the interactions with the meta-learning based adaptation compared to the previously studied statistical adaptation model.

show abstract

“…Several works have shown its use in training of an agent for behaviors similar to that of humans. The works by Qureshi et al [12] [13] presented an RL method for training an agent to greet as humans with the sequential actions of wait, look, wave and shake hand. They used multi-modal DQN and generated rewards at every successful handshake.…”

Section: Social Robots and Rlmentioning

confidence: 99%

Batch Recurrent Q-Learning for Backchannel Generation Towards Engaging Agents

Hussain

Erzin

Sezgin

et al. 2019

2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII)

View full text Add to dashboard Cite

The ability to generate appropriate verbal and non-verbal backchannels by an agent during humanrobot interaction greatly enhances the interaction experience. Backchannels are particularly important in applications like tutoring and counseling, which require constant attention and engagement of the user. We present here a method for training a robot for backchannel generation during a human-robot interaction within the reinforcement learning (RL) framework, with the goal of maintaining high engagement level. Since online learning by interaction with a human is highly time-consuming and impractical, we take advantage of the recorded human-to-human dataset and approach our problem as a batch reinforcement learning problem. The dataset is utilized as a batch data acquired by some behavior policy. We perform experiments with laughs as a backchannel and train an agent with value-based techniques. In particular, we demonstrate the effectiveness of recurrent layers in the approximate value function for this problem, that boosts the performance in partially observable environments. With off-policy policy evaluation, it is shown that the RL agents are expected to produce more engagement than an agent trained from imitation learning.Keywords human-robot interaction · engagement · partially observable Markov decision process · batch reinforcement learning

show abstract

Robot gains social intelligence through multimodal deep reinforcement learning

Cited by 89 publications

References 16 publications

Exploring Temporal Dependencies in Multimodal Referring Expressions with Mixed Reality

Exploring Temporal Dependencies in Multimodal Referring Expressions with Mixed Reality

Fast Adaptation with Meta-Reinforcement Learning for Trust Modelling in Human-Robot Interaction

Batch Recurrent Q-Learning for Backchannel Generation Towards Engaging Agents

Contact Info

Product

Resources

About