Investigating spatial knowledge acquisition in virtual environments allows studying different sources of information under controlled conditions. Therefore, we built a virtual environment in the style of a European village and investigated spatial knowledge acquisition by experience in the immersive virtual environment and compared it to using an interactive map of the same environment. The environment was well explored, with both exploration sources covering the whole village area. We tested knowledge of cardinal directions, building-to-building orientation, and judgment of direction between buildings in a pointing task. The judgment of directions was more accurate after exploration of the virtual environment than after map exploration. The opposite results were observed for knowledge of cardinal directions and relative orientation between buildings. Time for cognitive reasoning improved task accuracies after both exploration sources. Further, an alignment effect toward the north was only visible after map exploration. Taken together, our results suggest that the source of spatial exploration differentially influenced spatial knowledge acquisition.

Visual attention is mainly goal directed and allocated based on the upcoming action. However, it is unclear how far this feature of gaze behaviour generalizes in more naturalistic settings. The present study investigates the influence of action affordances on active inference processes revealed by eye movements during interaction with familiar and novel tools. In a between-subject design, a cohort of participants interacted with a virtual reality controller in a lowrealism environment; another performed the task with an interaction setup that allowed differentiated hand and finger movements in a high-realism environment. We investigated the differences in odds of fixations and their eccentricity towards the tool parts before action initiation. The results show that participants fixate more on the tool's effector part before action initiation when asked to produce tool-specific movements, especially with unfamiliar tools. These findings suggest that fixations are made in a task-oriented way to plan the distal goals of producing the task-and tool-specific actions well before action initiation. Moreover, with more realistic action affordance, fixations were biased towards the tool handle when it was oriented incongruent with the subjects' handedness. We hypothesize that these fixations are made towards the proximal goal of planning the grasp even though the perceived action on the tools is identical for both experimental setups. Taken together, proximal and distal goal-oriented planning is contextualized to the realism of action/interaction afforded by an environment.

No abstract

Seminal studies on human cognitive behavior have been conducted in controlled laboratory settings, demonstrating that visual attention is mainly goal-directed and allocated based on the action performed. However, it is unclear how far these results generalize to cognition in more naturalistic settings. The present study investigates active inference processes revealed by eye movements during interaction with familiar and novel tools with two levels of realism of the performed action. We presented participants with 3D tool models that were either familiar or unfamiliar, oriented congruent or incongruent to their handedness, and asked participants to interact with them by either lifting or using. Importantly, we used the same experimental design in two setups. In the first experiment, participants interacted with a VR controller; in the second, they performed the task with a more realistic interaction setup that allowed differentiated hand and finger movements. We used linear mixed models to determine the odds of fixations on the tool effector vs. handle before action initiation. The results show that participants fixate more on the tool's effector part before action initiation during the use task for unfamiliar tools. Furthermore, under more natural interaction conditions, subjects fixate more on the tool's handle as a function of the tool's orientation, well before the action was executed. Secondly, the spatial viewing bias on the tool reveals early fixations are associated with the task and familiarity of the tools. In contrast, later fixations are associated with the manual planning of the interaction. In sum, the findings from the experiments suggest that fixations are made in a task-oriented way to extract the mechanical properties of the tools to plan the intended action well before action initiation. Further, in more natural conditions, fixations are made towards proximal goals of optimally planning the grasp even when the perceived action on the tools is identical. Thus, the realism of the action in a virtual environment seems to be as important as the sensory stimulation.

Visual attention is mainly goal-directed and allocated based on the upcoming action to be performed. However, it is unclear how far this feature of gaze behavior generalizes in more naturalistic settings. The present study investigates active inference processes revealed by eye movements during interaction with familiar and novel tools with two levels of realism of the action affordance. In a between-subject design, a cohort of participants interacted with a VR controller in a low realism environment; another performed the task with an interaction setup that allowed differentiated hand and finger movements in a high-realism environment. We investigated the differences in odds of fixations and their eccentricity towards the tool parts before action initiation. The results show that participants fixate more on the tool’s effector part before action initiation when asked to use the tool and during interaction with unfamiliar tools. The spatial viewing bias on the tool reveals early fixations are influenced by the task and the familiarity of the tools. Our findings suggest that fixations are made in a task-oriented way to plan the intended action well before action initiation. With more realistic action affordances, more fixations were allocated toward the tool handle. We hypothesize that these fixations are made towards the proximal goal of planning the grasp even though the perceived action on the tools is identical for both experimental setups. Taken together, proximal and distal goal-oriented planning is contextualized to the realism of action/interaction afforded by an environment.