Anticipating human actions for collaboration in the presence of task and sensor uncertainty

Abstract

A representation for structured activities is developed that allows a robot to probabilistically infer which task actions a human is currently performing and to predict which future actions will be executed and when they will occur. The goal is to enable a robot to anticipate collaborative actions in the presence of uncertain sensing and task ambiguity. The system can represent multi-path tasks where the task variations may contain partially ordered actions or even optional actions that may be skipped altogether. The task is represented by an AND-OR tree structure from which a probabilistic graphical model is constructed. Inference methods for that model are derived that support a planning and execution system for the robot which attempts to minimize a cost function based upon expected human idle time. We demonstrate the theory in both simulation and actual human-robot performance of a two-waybranch assembly task. In particular we show that the inference model can robustly anticipate the actions of the human even in the presence of unreliable or noisy detections because of its integration of all its sensing information along with knowledge of task structure.

The HRC Model Set for Human-Robot Collaboration Research

“…Consequently, it is possible to devise an experiment in which there is a clear compartmentalization of the roles the two partners have. Similarly to [38], [40], [1], specific actions (e.g. bringing components) can be assigned to the robot, whereas the human can be entrusted to perform only the actions that require fine manipulation skills (e.g.…”

Fast target prediction of human reaching motion for cooperative human-robot manipulation tasks using time series classification

“…These works analyze the discrete action sequence of human and robot task steps [9][10][11] [12]; however, they do not take advantage of geometric or kinematic information. Approaches include the prediction of subsequent human action through inference in a Bayesian network [9], and consideration of task ambiguity and sensor uncertainty [10] during an assembly task similar to ours. Other techniques, such as simple temporal problems (STP) and nonlinear programming, have been used to produce optimal flexible schedules for human-robot collaboration [11].…”

Enabling Synchronous Joint Action in Human-Robot Teams

“…Joint action is one naturally present high-level social signal that occurs when multiple people coordinate their actions in time and space to make a change to their environment [11]. In the HRI community, researchers have been working in the area of joint action, where a robot can interpret and predict the intentions of groups to better inform actions, and lead to more fluent interactions [12,3,2,9,1,7,8,5,6].…”

“…al. [2] uses a probabilistic model that accounts for human inconsistencies to determine appropriate actions for a robot providing assembly parts. Further, Hoffman et.…”