Learning with bounded memory in games

Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction

Nath

Procaccia

et al. 2017

In human-robot teams, humans often start with an inaccurate model of the robot capabilities. As they interact with the robot, they infer the robot's capabilities and partially adapt to the robot, i.e., they might change their actions based on the observed outcomes and the robot's actions, without replicating the robot's policy. We present a game-theoretic model of human partial adaptation to the robot, where the human responds to the robot's actions by maximizing a reward function that changes stochastically over time, capturing the evolution of their expectations of the robot's capabilities. The robot can then use this model to decide optimally between taking actions that reveal its capabilities to the human and taking the best action given the information that the human currently has. We prove that under certain observability assumptions, the optimal policy can be computed efficiently. We demonstrate through a human subject experiment that the proposed model significantly improves human-robot team performance, compared to policies that assume complete adaptation of the human to the robot

Section: Bounded-memory Adaptation Modelmentioning

confidence: 99%

Game-Theoretic Modeling of Human Adaptation in Human-Robot Collaboration

Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction

Nath

Procaccia

et al. 2017

“…This assumption of "bounded rationality" was first proposed by Herbert Simon: people often do not have the time and cognitive capabilities to make perfectly rational decisions (Simon, 1979). In game theory, bounded rationality has been modeled by assuming that players have a "bounded memory" or "bounded recall" and base their decisions on recent observations (Powers & Shoham, 2005;Monte, 2014;Aumann & Sorin, 1989). We find this assumption particularly pertinent in a fast-paced task, such as the table carrying task, where the human collaborator has limited time to choose their actions.…”

Section: Bounded-memory Adaptation Modelmentioning

confidence: 99%

Planning with Verbal Communication for Human-Robot Collaboration

J. Hum.-Robot Interact.

Kwon

Forlizzi

et al. 2018

Human collaborators coordinate effectively their actions through both verbal and non-verbal communication. We believe that the the same should hold for human-robot teams. We propose a formalism that enables a robot to decide optimally between doing a task and issuing an utterance. We focus on two types of utterances: verbal commands, where the robot expresses how it wants its human teammate to behave, and state-conveying actions, where the robot explains why it is behaving this way. Human subject experiments show that enabling the robot to issue verbal commands is the most effective form of communicating objectives, while retaining user trust in the robot. Communicating why information should be done judiciously, since many participants questioned the truthfulness of the robot statements.

“…This is particularly true in the human-robot team setting, when the human is unsure about how the robot will act and has little time to respond. We propose a model of human adaptive behaviors based on a bounded memory assumption (Powers and Shoham, 2005; Monte, 2014; Aumann and Sorin, 1989) and integrate it into robot decision making.…”

Section: Relevant Workmentioning

confidence: 99%

“…This idea has been supported by studies in psychology and economics (Kahneman, 2003). In game theory, bounded rationality has been modeled by assuming that players have a “bounded memory” or “bounded recall” and base their decisions on recent observations (Powers and Shoham, 2005; Monte, 2014; Aumann and Sorin, 1989). In this work, we introduce the bounded memory assumption in a human-robot collaboration setting.…”

Section: The Bounded Memory Adaptation Modelmentioning

confidence: 99%

Human-robot mutual adaptation in collaborative tasks: Models and experiments

The International Journal of Robotics Research

Hsu

Srinivasa

2017

149

Adaptation is critical for effective team collaboration. This paper introduces a computational formalism for mutual adaptation between a robot and a human in collaborative tasks. We propose the Bounded-Memory Adaptation Model, which is a probabilistic finite-state controller that captures human adaptive behaviors under a bounded-memory assumption. We integrate the Bounded-Memory Adaptation Model into a probabilistic decision process, enabling the robot to guide adaptable participants towards a better way of completing the task. Human subject experiments suggest that the proposed formalism improves the effectiveness of human-robot teams in collaborative tasks, when compared with one-way adaptations of the robot to the human, while maintaining the human’s trust in the robot.