The very limited capacity of short-term or working memory is one of the most prominent features of human cognition. Most studies have stressed delimiting the upper bounds of this memory in memorization tasks rather than the performance of everyday tasks. We designed a series of experiments to test the use of short-term memory in the course of a natural hand-eye task where subjects have the freedom to choose their own task parameters. In this case subjects choose not to operate at the maximum capacity of short-term memory but instead seek to minimize its use. In particular, reducing the instantaneous memory required to perform the task can be done by serializing the task with eye movements. These eye movements allow subjects to postpone the gathering of task-relevant information until just before it is required. The reluctance to use short-term memory can be explained if such memory is expensive to use with respect to the cost of the serializing strategy.

This paper investigates the temporal dependencies of natural vision by measuring eye and hand movements while subjects made a sandwich. The phenomenon of change blindness suggests these temporal dependencies might be limited. Our observations are largely consistent with this, suggesting that much natural vision can be accomplished with "just-in-time" representations. However, we also observe several aspects of performance that point to the need for some representation of the spatial structure of the scene that is built up over different fixations. Patterns of eye-hand coordination and fixation sequences suggest the need for planning and coordinating movements over a period of a few seconds. This planning must be in a coordinate frame that is independent of eye position, and thus requires a representation of the spatial structure in a scene that is built up over different fixations.

While we know a great deal about the dynamics and characteristics of eye movements in relatively simple tasks performed under reduced laboratory conditions, we know less about oculomotor behavior in complex, multi-step tasks. Complex tasks are not necessarily difficult. Part of the transition from 'hard' to 'easy' in completing complex tasks is the gradual reduction in conscious effort required to complete the sub-tasks. We are interested in learning whether high-level perceptual strategies can aid that transition. In the past, subjects performed relatively simple tasks or the eye movements themselves were the instructed task. But outside the laboratory vision is a tool, not the task. To study the oculomotor system in its native mode, we developed a wearable eyetracker that allows natural eye, head and whole-body movements. Using the over-learned, common task of hand-washing, we measured the global characteristics of fixation duration, saccade amplitude, and the spatial distribution of fixation positions. An important observation was the emergence of higher-order perceptual strategies in the complex task: while most fixations were related to the immediate action, a small number of fixations were made to objects relevant only to future actions. Based on a control task that differed only in the high-level goal, we conclude that the look-ahead fixations represent a task-dependent strategy, not a general behavior elicited by the salience or conspicuity of objects in the environment. We propose that the strategy of looking ahead to objects of future relevance supports the conscious percept of an environment seamless in time as well as in space.

Relatively little is known about movements of the eyes, head, and hands in natural tasks. Normal behavior requires spatial and temporal coordination of the movements in more complex circumstances than are typically studied, and usually provides the opportunity for motor planning. Previous studies of natural tasks have indicated that the parameters of eye and head movements are set by global task constraints. In this experiment, we explore the temporal coordination of eye, head, and hand movements while subjects performed a simple block-copying task. The task involved fixations to gather information about the pattern, as well as visually guided hand movements to pick up and place blocks. Subjects used rhythmic patterns of eye, head, and hand movements in a fixed temporal sequence or coordinative structure. However, the pattern varied according to the immediate task context. Coordination was maintained by delaying the hand movements until the eye was available for guiding the movement. This suggests that observers maintain coordination by setting up a temporary, task-specific synergy between the eye and hand. Head movements displayed considerable flexibility and frequently diverged from the gaze change, appearing instead to be linked to the hand trajectories. This indicates that the coordination of eye and head in gaze changes is usually the consequence of a synergistic linkage rather than an obligatory one. These temporary synergies simplify the coordination problem by reducing the number of control variables, and consequently the attentional demands, necessary for the task.

No abstract