Representing 3D Space in Working Memory: Spatial Images from Vision, Hearing, Touch, and Language

Abstract

The chapter deals with a form of transient spatial representation referred to as a spatial image. Like a percept, it is externalized, scaled to the environment, and can appear in any direction about the observer. It transcends the concept of modality, as it can be based on inputs from the three spatial senses, from language, and from long-term memory. Evidence is presented that supports each of the claimed properties of the spatial image, showing that it is quite different from a visual image. Much of the evidence presented is based on spatial updating. A major concern is whether spatial images from different input modalities are functionally equivalentthat once instantiated in working memory, the spatial images from different modalities have the same functional characteristics with respect to subsequent processing, such as that involved in spatial updating. Going further, the research provides some evidence that spatial images are amodal (i.e., do not retain modality-speci fi c features).

Updating spatial relations to remote locations described in narratives

“…However, even in the absence of visualization instructions, some readers may also be compelled to use their sensorimotor framework to encode described locations, in particular when the situation affords salient correspondences between the described environment and the immediate/perceptual surroundings. These findings are in line with those from studies examining updating with perceptual scenes, supporting the idea that various input modalities give rise to spatial images that can be used to support behavior the same way regardless of the input (see Loomis, Klatzky, & Giudice 2013, for a review). 1.…”

Integration of spatial information across vision and language

“…Behavioral and neuroimaging findings suggest that spatial S t e p h a n i e N . P a n t e l i d e s learning from language is not different from that taking place through direct modalities such as vision and touch (for reviews see Loomis, Klatzky, Avraamides, Lippa, & Golledge, 2007;Loomis, Klatzky, & Giudice, 2013;Avraamides, Mello, & Greenauer, 2013). In fact, a number of theories in spatial cognition claimed that at some point, spatial memories acquired from different modalities are represented in a similar format (e.g., Bryant, 1997;Jackendoff, 1987;Miller & Johnson-laird, 1976).…”

More than just perception–action recalibration: Walking through a virtual environment causes rescaling of perceived space

“…An alternative formulation of the explicit strategy is the representation modification strategy, whereby the perceptual representation itself is explicitly modified to correct for underperception of egocentric distance. Explicit modification of the percept itself may be unlikely, but modification of an internal representation that persists after vision is occluded (like the "spatial image" described by Loomis, Klatzky, & Giudice, 2013) seems more plausible. Since the representation modification strategy would result in a new, corrected representation, any response based on the new representation should reflect the same correction that was initially applied, which could explain findings of generalization across response types (e.g., from direct walking responses to indirect walking responses; Richardson & Waller, 2007).…”