I Can See Your Aim: Estimating User Attention from Gaze for Handheld Robot Collaboration

Abstract

This paper explores the estimation of user attention in the setting of a cooperative handheld robot -a robot designed to behave as a handheld tool but that has levels of task knowledge. We use a tool-mounted gaze tracking system, which after modelling via a pilot study, we use as a proxy for estimating the attention of the user. This information is then used for cooperation with users in a task of selecting and engaging with objects on a dynamic screen. Via a video game setup, we test various degrees of robot autonomy from fully autonomous, where the robot knows what it has to do and acts, to no autonomy where the user is in full control of the task. Our results measure performance and subjective metrics and show how the attention model benefits the interaction and preference of users.

Towards Intention Prediction for Handheld Robots: a Case of Simulated Block Copying

“…Efforts towards involving user perception in the robot's task planning were made in our recent work on estimating user attention [3]. The method was inspired by work from Land et al who found that human's eye gaze is closely related to their manual actions [8].…”

“…2: The Handheld robot used for our study. It features a set of input buttons and a trigger at the handle, a 6-DoF tip and user perception through gaze tracking as reported in [3].…”

“…As a starting point of addressing this problem, extended user perception was introduced to the robot which allows the estimation of the user's eye gaze in 3D space during task execution [3]. An estimate of users' visual attention was then used to inform the robot's decisions when there was an alternative e.g.…”

“…For our study, we use the open robotic platform 1 , introduced in [7] in combination with an eye tracking system as reported in [3]. Within a simulated assembly task, eye gaze information is used to predict subsequent user actions.…”

Rebellion and Obedience: The Effects of Intention Prediction in Cooperative Handheld Robots

“…As a starting point of addressing this problem, extended user perception can be introduced to allow the robot to estimate the user's point of attention via eye gaze in 3D space during task execution [3]. An estimate of users' visual attention informs the robot about areas of users' interest.…”