The role of memory in guiding attention allocation in daily behaviors is not well understood. In experiments with two-dimensional (2D) images, there is mixed evidence about the importance of memory. Because the stimulus context in laboratory experiments and daily behaviors differs extensively, we investigated the role of memory in visual search, in both two-dimensional (2D) and three-dimensional (3D) environments. A 3D immersive virtual apartment composed of two rooms was created, and a parallel 2D visual search experiment composed of snapshots from the 3D environment was developed. Eye movements were tracked in both experiments. Repeated searches for geometric objects were performed to assess the role of spatial memory. Subsequently, subjects searched for realistic context objects to test for incidental learning. Our results show that subjects learned the room-target associations in 3D but less so in 2D. Gaze was increasingly restricted to relevant regions of the room with experience in both settings. Search for local contextual objects, however, was not facilitated by early experience. Incidental fixations to context objects do not necessarily benefit search performance. Together, these results demonstrate that memory for global aspects of the environment guides search by restricting allocation of attention to likely regions, whereas task relevance determines what is learned from the active search experience. Behaviors in 2D and 3D environments are comparable, although there is greater use of memory in 3D.

Previous work on transsaccadic memory and change blindness suggests that only a small part of the information in the visual scene is retained following a change in eye position. However, some visual representation across different fixation positions seems necessary to guide body movements. To understand what information is retained across gaze positions, it seems necessary to consider the functional demands of vision in ordinary behavior. We therefore examined eye and hand movements in a naturalistic task, where subjects copied a toy model in a virtual environment. Saccadic targeting performance was examined to see if subjects took advantage of regularities in the environment. During the first trials the spatial arrangement of the pieces used to copy the model was kept stable. In subsequent trials this arrangement was changed randomly every time the subject looked away. Results showed that about 20% of saccades went either directly to the location of the next component to be copied or to its old location before the change. There was also a significant increase in the total number of fixations required to locate a piece after a change, which could be accounted for by the corrective movements required after fixating the (incorrect) old location. These results support the idea that a detailed representation of the spatial structure of the environment is typically retained across fixations and used to guide eye movements.

To successfully move our hand to a target, we must consider how to get there without hitting surrounding objects. In a dynamic environment this involves being able to respond quickly when our relationship with surrounding objects changes. People adjust their hand movements with a latency of about 120 ms when the visually perceived position of their hand or of the target suddenly changes. It is not known whether people can react as quickly when the position of an obstacle changes. Here we show that quick responses of the hand to changes in obstacle position are possible, but that these responses are direct reactions to the motion in the surrounding. True adjustments to the changed position of the obstacle appeared at much longer latencies (about 200 ms). This is even so when the possible change is predictable. Apparently, our brain uses certain information exceptionally quickly for guiding our movements, at the expense of not always responding adequately. For reaching a target that changes position, one must at some time move in the same direction as the target did. For avoiding obstacles that change position, moving in the same direction as the obstacle is not always an adequate response, not only because it may be easier to avoid the obstacle by moving the other way, but also because one wants to hit the target after passing the obstacle. Perhaps subjects nevertheless quickly respond in the direction of motion because this helps avoid collisions when pressed for time.