Spatial working memory for locations specified by vision and audition: Testing the amodality hypothesis

Loomis, Jack M.; Klatzky, Roberta L.; McHugh, Brendan; Giudice, Nicholas A.

doi:10.3758/s13414-012-0311-2

Cited by 25 publications

(47 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The present results support the fact that short-term-memory distortions may have affected the localization performance. The results also speak against the amodality hypothesis (i.e., spatial images have no trace of their modal origins, Loomis et al, 2012 ) because the patterns of responses clearly reveal the initial coding of the stimuli.…”

Section: Discussionmentioning

confidence: 69%

The interaction of vision and audition in two-dimensional space

et al. 2015

View full text Add to dashboard Cite

Using a mouse-driven visual pointer, 10 participants made repeated open-loop egocentric localizations of memorized visual, auditory, and combined visual-auditory targets projected randomly across the two-dimensional frontal field (2D). The results are reported in terms of variable error, constant error and local distortion. The results confirmed that auditory and visual maps of the egocentric space differ in their precision (variable error) and accuracy (constant error), both from one another and as a function of eccentricity and direction within a given modality. These differences were used, in turn, to make predictions about the precision and accuracy within which spatially and temporally congruent bimodal visual-auditory targets are localized. Overall, the improvement in precision for bimodal relative to the best unimodal target revealed the presence of optimal integration well-predicted by the Maximum Likelihood Estimation (MLE) model. Conversely, the hypothesis that accuracy in localizing the bimodal visual-auditory targets would represent a compromise between auditory and visual performance in favor of the most precise modality was rejected. Instead, the bimodal accuracy was found to be equivalent to or to exceed that of the best unimodal condition. Finally, we described how the different types of errors could be used to identify properties of the internal representations and coordinate transformations within the central nervous system (CNS). The results provide some insight into the structure of the underlying sensorimotor processes employed by the brain and confirm the usefulness of capitalizing on naturally occurring differences between vision and audition to better understand their interaction and their contribution to multimodal perception.

show abstract

Section: Discussionmentioning

confidence: 69%

The interaction of vision and audition in two-dimensional space

et al. 2015

View full text Add to dashboard Cite

show abstract

“…Another alternative is that participants may not have access to modality-specific aspects of their experience in memory, but that an amodal sense of studied space was weighted fully on the more “conservative” sense (in this case “haptics”). Similar alternatives have been raised in research on visual and auditory spatial working memory; and it has been suggested that shorter retention intervals may be associated with memory for modal information with more long-term representations relying on an amodal code, but this has not yet been fully resolved (e.g., Giudice, Klatzky & Loomis, 2009; Loomis, Klatzky, McHugh, & Giudice, 2012). It is difficult to make direct comparisons across these experiments and the current experiments given the differences in stimuli, task and modality, but clearly the same issues apply.…”

Section: Discussionmentioning

confidence: 74%

“…The unimodal and cross-modal results of Experiment 1 suggests that this framework can also support multiple sensory inputs (vision and haptics). In important ways, this framework is similar to the “spatial image” described in Loomis, Klatzky, & Giudice’s (2012) discussion of memory for multimodal stimuli in that it is a surrounding spatial structure (not limited to the frontal plane) and is an amodal spatial structure. The difference is that the “spatial image” is instantiated in working memory; whereas the spatial structure in the multisource model is conceptualized as a “standing” mental structure that provides the basis of scene perception.…”

Section: Discussionmentioning

confidence: 98%

“…The idea is that the observer brings to any view of a scene this sense of surrounding space and top-down expectations incorporated into this spatial structure provide anticipations about upcoming space. The idea of an amodal structure is similar to the “spatial image” proposed by Loomis, Klatzky, & Giudice (2012), in that it is a surrounding spatial representation (not limited to the frontal plane) and that it is amodal. The only difference is that unlike the “spatial image”, the spatial structure in the multisource model is conceptualized as a “standing framework”, rather than one that develops in response to a stimulus in working memory.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Visual, haptic and bimodal scene perception: Evidence for a unitary representation

2015

View full text Add to dashboard Cite

Participants studied seven meaningful scene-regions bordered by removable boundaries (30 s each). In Experiment 1 (N=80) participants used visual or haptic exploration and then minutes later, reconstructed boundary position using the same or the alternate modality. Participants in all groups shifted boundary placement outward (boundary extension), but visual study yielded the greater error. Critically, this modality-specific difference in boundary extension transferred without cost in the cross-modal conditions, suggesting a functionally unitary scene representation. In Experiment 2 (N= 20), bimodal study led to boundary extension that did not differ from haptic exploration alone, suggesting that bimodal spatial memory was constrained by the more “conservative” haptic modality. In Experiment 3 (N=20), as in picture studies, boundary memory was tested 30 s after viewing each scene-region and as with pictures, boundary extension still occurred. Results suggest that scene representation is organized around an amodal spatial core that organizes bottom-up information from multiple modalities in combination with top-down expectations about the surrounding world.

show abstract

“…As such, perceptual errors in distance or direction are inherited by the spatial image and remain once the stimulus and resulting percept are no longer present. There is growing evidence that the spatial image is amodal in nature--that once formed, it no longer retains modality-specific information with respect to subsequent tasks that rely on it (Giudice, Betty, and Loomis, 2011; Giudice, Klatzky, and Loomis, 2009; Loomis et al, 2012; Loomis et al, in press). In this article, our interest is whether there are spatial images of the perceived locations of targets sensed by extended touch (i.e., using a probe to extend the reach of the arm) as there are with normal touch (e.g., Giudice et al, 2011) and, if so, to assess the accuracy of extended touch relative to normal touch and vision.…”

Section: Introductionmentioning

confidence: 99%

Perception of 3-D location based on vision, touch, and extended touch

et al. 2012

Self Cite

View full text Add to dashboard Cite

Perception of the near environment gives rise to spatial images in working memory that continue to represent the spatial layout even after cessation of sensory input. As the observer moves, these spatial images are continuously updated.This research is concerned with (1) whether spatial images of targets are formed when they are sensed using extended touch (i.e., using a probe to extend the reach of the arm) and (2) the accuracy with which such targets are perceived. In Experiment 1, participants perceived the 3-D locations of individual targets from a fixed origin and were then tested with an updating task involving blindfolded walking followed by placement of the hand at the remembered target location. Twenty-four target locations, representing all combinations of two distances, two heights, and six azimuths, were perceived by vision or by blindfolded exploration with the bare hand, a 1-m probe, or a 2-m probe. Systematic errors in azimuth were observed for all targets, reflecting errors in representing the target locations and updating. Overall, updating after visual perception was best, but the quantitative differences between conditions were small. Experiment 2 demonstrated that auditory information signifying contact with the target was not a factor. Overall, the results indicate that 3-D spatial images can be formed of targets sensed by extended touch and that perception by extended touch, even out to 1.75 m, is surprisingly accurate.

show abstract

Spatial working memory for locations specified by vision and audition: Testing the amodality hypothesis

Cited by 25 publications

References 39 publications

The interaction of vision and audition in two-dimensional space

The interaction of vision and audition in two-dimensional space

Visual, haptic and bimodal scene perception: Evidence for a unitary representation

Perception of 3-D location based on vision, touch, and extended touch

Contact Info

Product

Resources

About