When interacting with our environment we generally make use of egocentric and allocentric object information by coding object positions relative to the observer or relative to the environment, respectively. Bayesian theories suggest that the brain integrates both sources of information optimally for perception and action. However, experimental evidence for egocentric and allocentric integration is sparse and has only been studied using abstract stimuli lacking ecological relevance. Here, we investigated the use of egocentric and allocentric information during memory-guided reaching to images of naturalistic scenes. Participants encoded a breakfast scene containing six objects on a table (local objects) and three objects in the environment (global objects). After a 2 s delay, a visual test scene reappeared for 1 s in which 1 local object was missing (= target) and of the remaining, 1, 3 or 5 local objects or one of the global objects were shifted to the left or to the right. The offset of the test scene prompted participants to reach to the target as precisely as possible. Only local objects served as potential reach targets and thus were task-relevant. When shifting objects we predicted accurate reaching if participants only used egocentric coding of object position and systematic shifts of reach endpoints if allocentric information were used for movement planning. We found that reaching movements were largely affected by allocentric shifts showing an increase in endpoint errors in the direction of object shifts with the number of local objects shifted. No effect occurred when one local or one global object was shifted. Our findings suggest that allocentric cues are indeed used by the brain for memory-guided reaching towards targets in naturalistic visual scenes. Moreover, the integration of egocentric and allocentric object information seems to depend on the extent of changes in the scene.

Numerous studies have demonstrated that humans incorporate allocentric information when reaching toward visual targets. So far, it is unclear how this information is integrated into the movement plan when multiple allocentric cues are available. In this study we investigated whether and how the extent of spatial changes and the task relevance of allocentric cues influence reach behavior. To this end, we conducted two experiments where we presented participants three-dimensional-rendered images of a naturalistic breakfast scene on a computer screen. The breakfast scene included multiple objects (allocentric cues) with a subset of objects functioning as potential reach targets (i.e., they were task-relevant). Participants freely viewed the scene and after a short delay, the scene reappeared with one object missing (target) and other objects being shifted left- or rightwards. Afterwards, participants were asked to reach toward the target position on a gray screen while fixating the screen center. We found systematic deviations of reach endpoints in the direction of object shifts which varied with the number of objects shifted, but only if these objects served as potential reach targets. Our results suggest that the integration of allocentric information into the reach plan is determined by contextual factors, in particular by the extent of spatial cue changes and the task-relevance of allocentric cues.

Previous research has demonstrated that humans use allocentric information when reaching to remembered visual targets, but most of the studies are limited to 2D space. Here, we study allocentric coding of memorized reach targets in 3D virtual reality. In particular, we investigated the use of allocentric information for memory-guided reaching in depth and the role of binocular and monocular (object size) depth cues for coding object locations in 3D space. To this end, we presented a scene with objects on a table which were located at different distances from the observer and served as reach targets or allocentric cues. After free visual exploration of this scene and a short delay the scene reappeared, but with one object missing (=reach target). In addition, the remaining objects were shifted horizontally or in depth. When objects were shifted in depth, we also independently manipulated object size by either magnifying or reducing their size. After the scene vanished, participants reached to the remembered target location on the blank table. Reaching endpoints deviated systematically in the direction of object shifts, similar to our previous results from 2D presentations. This deviation was stronger for object shifts in depth than in the horizontal plane and independent of observer-target-distance. Reaching endpoints systematically varied with changes in object size. Our results suggest that allocentric information is used for coding targets for memory-guided reaching in depth. Thereby, retinal disparity and vergence as well as object size provide important binocular and monocular depth cues.

Previous research has shown that egocentric and allocentric information is used for coding target locations for memory-guided reaching movements. Especially, task-relevance determines the use of objects as allocentric cues. Here, we investigated the influence of scene configuration and object reliability as a function of task-relevance on allocentric coding for memory-guided reaching. For that purpose, we presented participants images of a naturalistic breakfast scene with five objects on a table and six objects in the background. Six of these objects served as potential reach-targets (= task-relevant objects). Participants explored the scene and after a short delay, a test scene appeared with one of the task-relevant objects missing, indicating the location of the reach target. After the test scene vanished, participants performed a memory-guided reaching movement toward the target location. Besides removing one object from the test scene, we also shifted the remaining task-relevant and/or task-irrelevant objects left- or rightwards either coherently in the same direction or incoherently in opposite directions. By varying object coherence, we manipulated the reliability of task-relevant and task-irrelevant objects in the scene. In order to examine the influence of scene configuration (distributed vs. grouped arrangement of task-relevant objects) on allocentric coding, we compared the present data with our previously published data set (Klinghammer et al., 2015). We found that reaching errors systematically deviated in the direction of object shifts, but only when the objects were task-relevant and their reliability was high. However, this effect was substantially reduced when task-relevant objects were distributed across the scene leading to a larger target-cue distance compared to a grouped configuration. No deviations of reach endpoints were observed in conditions with shifts of only task-irrelevant objects or with low object reliability irrespective of task-relevancy. Moreover, when solely task-relevant objects were shifted incoherently, the variability of reaching endpoints increased compared to coherent shifts of task-relevant objects. Our results suggest that the use of allocentric information for coding targets for memory-guided reaching depends on the scene configuration, in particular the average distance of the reach target to task-relevant objects, and the reliability of task-relevant allocentric information.

In this study, we investigated the influence of gaze and prior knowledge about the reach target on the use of allocentric information for memory-guided reaching. Participants viewed a breakfast scene with five objects in the background and six objects on the table. Table objects served as potential reach targets. Participants first encoded the scene and, after a short delay, a test scene was presented with one table object missing and one, three, or five table objects horizontally shifted in the same direction. Participants performed a memory-guided reaching movement toward the position of the missing object on a blank screen. In order to examine the influence of gaze, participants either freely moved their gaze (free-view) or kept gaze at a fixation point (fixation) throughout the trial. The effect of prior knowledge was investigated by informing participants about the reach target either before (preview) or after (nonpreview) scene encoding. Our results demonstrate that humans use allocentric information for reaching even if a stable retinal reference is available. However, allocentric coding of reach targets is stronger when gaze is free and prior knowledge about the reach target is missing.