Memory shapes visual search strategies in large-scale environments

Abstract

Search is a central visual function. Most of what is known about search derives from experiments where subjects view 2D displays on computer monitors. In the natural world, however, search involves movement of the body in large-scale spatial contexts, and it is unclear how this might affect search strategies. In this experiment, we explore the nature of memory representations developed when searching in an immersive virtual environment. By manipulating target location, we demonstrate that search depends on episodic spatial memory as well as learnt spatial priors. Subjects rapidly learned the large-scale structure of the space, with shorter paths and less head rotation to find targets. These results suggest that spatial memory of the global structure allows a search strategy that involves efficient attention allocation based on the relevance of scene regions. Thus spatial memory may allow less energetically costly search strategies.

The Foreground Bias: Differing impacts across depth on visual search in scenes

“…Second, a number of studies have highlighted that spatial positioning plays a strong role in guiding search towards objects when finding objects in both 2D scene images (Castelhano & Heaven, 2011;Malcolm & Henderson, 2010;Pereira & Castelhano, 2019;Võ & Henderson, 2009;Williams & Castelhano, 2019) and virtual 3D environments (Hayhoe & Matthis, 2018;Kit et al, 2014;Li, Aivar, Kit, Tong, & Hayhoe, 2016;Li, Aivar, Tong, & Hayhoe, 2018). Thus, it seems reasonable that the depth at which the information occurs would also play a role when searching in a real environment.…”

Control of gaze in natural environments: effects of rewards and costs, uncertainty and memory in target selection

“…Whole body movements can also be minimized using spatial memory. Li et al [29] performed an experiment where subjects searched for targets in a virtual apartment. After searching for the target on three separate occasions, the target was moved to another location.…”

“…Subjects rapidly learn the layout of the environment and adopt shorter paths. Adapted from Li et al[29].…”

Learning in the eyes: specific changes in gaze patterns track explicit and implicit visual learning

“…These include observations that gaze biases can emerge from a lifetime of experience, such as taking the inherent uncertainty of the visual system into consideration during visual search (Najemnik & Geisler, 2005), anticipating a ball's trajectory in sports (Brockmole & Henderson, 2006;Land & McLeod, 2000), the tendency to perform visual search from left to right (Spalek & Hammad, 2005), using learnt semantic knowledge (Võ & Wolfe, 2013) or meaning in real world scenes (Henderson et al, 2018). At shorter time-scales, object cooccurrences (Brockmole & Henderson, 2006;Mack & Eckstein, 2011) and episodic memory have been shown to guide visual search (Li et al, 2018). Past experience on an even shorter time-scale can also influence gaze selection, for example when integrating visual information in a given scene with what has been learned about stimulus statistics within minutes (Hoppe & Rothkopf, 2016;Yang et al, 2017).…”

“…Past experience on an even shorter time-scale can also influence gaze selection, for example when integrating visual information in a given scene with what has been learned about stimulus statistics within minutes (Hoppe & Rothkopf, 2016;Yang et al, 2017). A number of these studies investigate jointly how humans develop specific eye movement patterns based on experience with the structure of sensory input and how they use specific eye movement strategies to solve particular tasks (Brockmole & Henderson, 2006;Hoppe & Rothkopf, 2016;Land & McLeod, 2000;Li et al, 2018;Mack & Eckstein, 2011;Nelson & Cottrell, 2007;Yang et al, 2017). However, all the above studies considered specific tasks (e.g.…”