Human Strategies for Multitasking, Search, and Control Improved via Real-Time Memory Aid for Gaze Location

Abstract

Objective: We aimed to elucidate how our domain-general cuing algorithm improved multitasking performance and changed behavioral strategies in human operators.Background: Though many gaze-control systems have been designed, previous realtime gaze-aware assistance systems were not both successful and domain-general. It is largely unknown what constitutes optimal search efficiency using the eyes, or ideal control using the mouse. It is unclear what the best coordinating strategies are between these two modalities. Our previously developed closed-loop multitasking aid drastically improved multitasking performance, though the behavioral mechanisms through which it acted were unknown. Methods:We performed in-depth analyses and generated novel eye tracking and mouse movement measures, to explore the complex effects of our helpful system on gaze and motor behavior.Results: Our overlay cuing algorithm improved control efficiency and reduced wellknown biases in search patterns. This system also reduced micromanaging behavior, with humans rationally relying more on imperfect automation in experimental assistance cue conditions. We showed that mouse and gaze were more independently specialized in the helpful cuing condition than in control conditions. Specifically, with our aid, the gaze performed more global movement, and the mouse performed more local clustered movement. Further, the gaze shifted toward search over processing with the helpful cuing system. We also illustrated a relationship between the mouse and the gaze, such that in these studies, "the hand was quicker than the eye." Conclusion:Overall, results suggested that our cuing system improved performance and reduced short-term working memory load on humans by delegating it to the computer in real time. Further, it reduced the number of required repeated decisions by an estimate of about one per second. It also enabled the gaze to specialize for improved visual search behavior, and the mouse to specialize for improved control.

EyeFrame: real-time memory aid improves human multitasking via domain-general eye tracking procedures

“…Below we detailed the practically relevant results with bearing for applications in human factors and related fields. For a cognitive and mechanistic deconstruction of the effect of our cuing system on human participants, we provided an additional elaborate analysis in Taylor et al (2015).…”

“…We provide an extended in-depth eye tracking analysis of these studies, as well as further discussion of prior assistive systems, divided attention, bias, supervisory sampling, search, task switching, working memory, augmented cognition, contingent eye tracking, gaze-control, gaze-aware displays, and other topics in Taylor et al (2015).…”

“…For those eye movements that cannot easily be intentionally controlled, salient "popout" cues (e.g., flashing red box around target) can reliably direct attention in a more automatic, bottom-up manner. As we discuss further, many eye tracking systems have been developed for realtime control, with very few attempting pure assistance, though none were both successful and domain-general (Taylor et al, 2015). There appears to be a need for such an assistive system.…”

The Effects of Transcranial Direct Current Stimulation (tDCS) on Multitasking Throughput Capacity

“…Rather than performing tasks simultaneously, multitasking involves shifting attention between tasks in a serial fashion (Hambrick et al, 2010; Adler and Benbunan-Fich, 2012). Working memory stores information regarding the tasks attended to most recently allowing the individual to predict which task requires attention next (Taylor et al, 2015a,b). Converging evidence suggests that tDCS applied to the DLPFC improves working memory performance.…”