Gaze-based intention estimation: principles, methodologies, and applications in HRI

Abstract

Intention prediction has become a relevant field of research in Human-Machine and Human-Robot Interaction. Indeed, any artificial system (co)-operating with and along humans, designed to assist and coordinate its actions with a human partner, would benefit from first inferring the human's current intention. To spare the user the cognitive burden of explicitly uttering their goals, this inference relies mostly on behavioral cues deemed indicative of the current action. It has been long known that eye movements are highly anticipatory of the single steps unfolding during a task, hence they can serve as a very early and reliable behavioural cue for intention recognition. This review aims to draw a line between insights in the psychological literature on visuomotor control and relevant applications of gaze-based intention recognition in technical domains, with a focus on teleoperated and assistive robotic systems.Starting from the cognitive principles underlying the relationship between intentions, eye movements, and action, the use of eye tracking and gaze-based models for intent recognition in Human-Robot Interaction is considered, with prevalent methodologies and their diverse applications. Finally, special consideration is given to relevant human factors issues and current limitations to be factored in when designing such systems.CCS Concepts: • General and reference → Surveys and overviews; • Computing methodologies → Cooperation and coordination;• Human-centered computing → HCI theory, concepts and models.

Integrating Egocentric and Robotic Vision for Object Identification Using Siamese Networks and Superquadric Estimations in Partial Occlusion Scenarios

“…Researchers use eyetracking devices to capture gaze data, which comprises the eye's position as 2D coordinates on a calibrated surface or within a scene. These data, crucial in revealing gaze events such as fixations and saccades, have been extensively studied, as noted in the work of Belardinelli et al [8]. To decode and understand these gaze data, various predictive models have been employed.…”

“…By projecting the gaze tracking onto the image seen by the user, an understanding of the user's visual intent is gained. Most work employing eye-tracking glasses in HRI focuses on gaze-based intention estimation for interactions like grasping with objects in fixed positions or known to the robot's object detector [8,9]. While interpreting gaze and intent is undeniably crucial, the subsequent challenge is substantial: translating this information into meaningful robot actions, especially when the robot lacks prior knowledge of the objects in its environment.…”

Gaze-Based Intention Recognition for Human-Robot Collaboration

“…Furthermore, when people are interacting with their surroundings or other individuals, their gaze frequently anticipate their movements [3] and as a result of this characteristic, eye tracking might be employed to foresee their intents. Precisely for this reasons, gaze estimation is a rapidly progressing area of research and finds application in a variety of fields: from the study of social interactions [6] to robotics [4,19]. Furthermore, in robotics, the importance of the gaze is twofold as it allows both the prediction of user actions [12,23,24] and the reproduction of credible behaviour on humanoid robots [1,8].…”