Learning behaviors via human-delivered discrete feedback: modeling implicit feedback strategies to speed up learning

Loftin, Robert; Peng, Bei; MacGlashan, James; Littman, Michael L.; Taylor, Matthew E.; Huang, Jeff; Roberts, David L.

doi:10.1007/s10458-015-9283-7

Cited by 68 publications

(89 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While the work mentioned above interprets human feedback as a numeric reward, Loftin et al [32] interpret human feedback as categorical feedback strategies that depend both on the behavior the trainer is trying to teach and the trainer's teaching strategy. Then they proposed an algorithm to infer knowledge about the desired behavior from cases where no feedback is provided.…”

Section: Reinforcement Learning From Human Rewardmentioning

confidence: 99%

Facial feedback for reinforcement learning: a case study and offline analysis using the TAMER framework

Dibeklioğlu

Whiteson

et al. 2020

Auton Agent Multi-Agent Syst

View full text Add to dashboard Cite

Interactive reinforcement learning provides a way for agents to learn to solve tasks from evaluative feedback provided by a human user. Previous research showed that humans give copious feedback early in training but very sparsely thereafter. In this article, we investigate the potential of agent learning from trainers' facial expressions via interpreting them as evaluative feedback. To do so, we implemented TAMER which is a popular interactive reinforcement learning method in a reinforcement-learning benchmark problem -Infinite Mario, and conducted the first large-scale study of TAMER involving 561 participants. With designed CNN-RNN model, our analysis shows that telling trainers to use facial expressions and competition can improve the accuracies for estimating positive and negative feedback using facial expressions. In addition, our results with a simulation experiment show that learning solely from predicted feedback based on facial expressions is possible and using strong/effective prediction models or a regression method, facial responses would significantly improve the performance of agents. Furthermore, our experiment supports previous studies demonstrating the importance of bi-directional feedback and competitive elements in the training interface.

show abstract

Section: Reinforcement Learning From Human Rewardmentioning

confidence: 99%

Facial feedback for reinforcement learning: a case study and offline analysis using the TAMER framework

Dibeklioğlu

Whiteson

et al. 2020

Auton Agent Multi-Agent Syst

View full text Add to dashboard Cite

show abstract

“…While the work mentioned above interprets human feedback as a numeric reward, Loftin et al [30] interpret human feedback as categorical feedback strategies that depend both on the behavior the trainer is trying to teach and the trainer's teaching strategy. They infer knowledge about the desired behavior from cases where no feedback is provided and show that their algorithms could learn faster than algorithms that treat the feedback as a numeric reward.…”

Section: Learning From Human Rewardmentioning

confidence: 99%

Social interaction for efficient agent learning from human reward

Whiteson

Knox

et al. 2017

Auton Agent Multi-Agent Syst

View full text Add to dashboard Cite

“…Including human in the loop has been recognized as an effective way to accelerate on-line policy learning (Thomaz and Breazeal, 2006;Khan et al, 2011;Cakmak and Lopes, 2012;Loftin et al, 2016). Most previous approaches employ teaching signals at the end of dialogues, either the whole human-to-human dialogue history or a single reward to evaluate the human-machine dialogue performance (Su et al, 2016;Ferreira and Lefèvre, 2015).…”

Section: Companion Teaching For On-line Dialogue Policy Learningmentioning

confidence: 99%

On-line Dialogue Policy Learning with Companion Teaching

Lü

Yang

Chang

et al. 2017

Proceedings of the 15th Conference of the European Chapter of The Association for Computational Linguistics: Volume 2

View full text Add to dashboard Cite

On-line dialogue policy learning is the key for building evolvable conversational agent in real world scenarios. Poor initial policy can easily lead to bad user experience and consequently fail to attract sufficient real users for policy training. We propose a novel framework, companion teaching, to include a human teacher in the on-line dialogue policy training loop to address the cold start problem. Here, dialogue policy is trained using not only user's reward but also teacher's example action as well as estimated immediate reward at turn level. Simulation experiments showed that, with a small number of human teaching dialogues, the proposed approach can effectively improve user experience at the beginning and smoothly lead to good performance with more user interaction data.

show abstract

Learning behaviors via human-delivered discrete feedback: modeling implicit feedback strategies to speed up learning

Cited by 68 publications

References 14 publications

Facial feedback for reinforcement learning: a case study and offline analysis using the TAMER framework

Facial feedback for reinforcement learning: a case study and offline analysis using the TAMER framework

Social interaction for efficient agent learning from human reward

On-line Dialogue Policy Learning with Companion Teaching

Contact Info

Product

Resources

About