Dynamics of eye-hand coordination are flexibly preserved in eye-cursor coordination during an online, digital, object interaction task

Abstract

Do patterns of eye-hand coordination observed during real-world object interactions apply to digital, screen-based object interactions? We adapted a real-world object interaction task (physically transferring cups in sequence about a tabletop) into a two-dimensional screen-based task (dragging-and-dropping circles in sequence with a cursor). We collected gaze (with webcam eye-tracking) and cursor position data from 51 fully-remote, crowd-sourced participants who performed the task on their own computer. We applied real-world time-series data segmentation strategies to resolve the self-paced movement sequence into phases of object interaction and rigorously cleaned the webcam eye-tracking data. In this preliminary investigation, we found that: 1) real-world eye-hand coordination patterns persist and adapt in this digital context, and 2) remote, online, cursor-tracking and webcam eye-tracking are useful tools for capturing visuomotor behaviours during this ecologically-valid human-computer interaction task. We discuss how these findings might inform design principles and further investigations into natural behaviours that persist in digital environments. CCS CONCEPTS• Human-centered computing → Empirical studies in HCI; Web-based interaction; User centered design; Laboratory experiments.

Continuous Measures of Decision-Difficulty Captured Remotely: II. Webcam eye-tracking reveals early decision processing

“…Using custom MATLAB scripts and our Gaze and Movement Analysis software, our initial data cleaning approach aimed to assess whether the gaze data showed reasonable patterns or whether it contained noisy, spurious gaze prediction errors (see [4] for a similar approach). Using more exaggerated but still mutuallyexclusive boundaries (see Figure 3B -AOI Dwell Boundaries), we determined whether the gaze fell inside at least one of the task-critical areas during the decision period.…”

“…A major drawback shared by all of the aforementioned eye-tracking studies is their confinement to laboratory settings. Recently, however, the use of webcam eye-tracking has emerged as a promising avenue in bridging the gap between controlled laboratory experiments and data collected in a wide range of environments (e.g., [49,47,4]). Admittedly, webcam eye-tracking is still a method in its infancy and has notable limitations in both temporal and spatial accuracy [40,3,49].…”