Commonalities of visual and auditory working memory in a spatial-updating task

Abstract

Although visual and auditory inputs are initially processed in separate perception systems, studies have built on the idea that to maintain spatial information these modalities share a component of working memory. The present study used working memory navigation tasks to examine functional similarities and dissimilarities in the performance of updating tasks. Participants mentally updated the spatial location of a target in a virtual array in response to sequential pictorial and sonant directional cues before identifying the target's final location. We predicted that if working memory representations are modality-specific, mixedmodality cues would demonstrate a cost of modality switching relative to unimodal cues. The results indicate that updating performance using visual unimodal cues positively correlated with that using auditory unimodal cues. Task performance using unimodal cues was comparable to that using mixed modality cues. The results of a subsequent experiment involving updating of target traces were consistent with those of the preceding experiments and support the view of modality-nonspecific memory.

Processing symmetry between visual and auditory spatial representations in updating working memory

“…Thirty students recruited from Hokkaido University (16 males; mean age = 21.0 years, range = 18-32 years) participated in Experiment 1A for monetary compensation or course credit. We chose this sample size to match the participant number in our previous study (Maezawa & Kawahara, 2021), which examined the effects of modality on working memory navigation. All participants had normal or corrected-to-normal visual acuity, and none reported any hearing impairment.…”

“…Thus, auditory input would be subdivided into non-spatial (i.e., what) and spatial (i.e., where) information (e.g., Alain et al, 2009;Delogu et al, 2012a). Consistent with this idea, some studies of spatial working memory (Lehnert & Zimmer, 2006;Loomis et al, 2012;Maezawa & Kawahara, 2021) have argued that visual and auditory spatial information are maintained and updated in a modality-general subsystem of working memory by recursive reactivation via controlled attention, similar to the attentional rehearsal system of the visuo-spatial sketchpad (Awh et al, 1998). Thus, extant studies have focused on the overlap of attentional resources across modalities.…”

“…Before addressing the main question about processing asymmetry between visual and auditory spatial working memory, we examined the similarity of these memory updating functions to identify modality-general attentional resources (Maezawa & Kawahara, 2021). Experiment 1A used the mental pathway task (Cornoldi et al, 1991;Kerr, 1993) in which participants maneuvered target locations using visual and auditory directional cues presented simultaneously.…”

“…Given this shared view of attentional resources in spatial working memory (Lehnert & Zimmer, 2008a;Martinkauppi et al, 2000), we have argued that visual and auditory representations would be updated relying on common processes (Maezawa & Kawahara, 2021). Importantly, this modality-general view also posits that bimodal stimuli, such as pairs of pictures and sounds appearing in temporal and spatial proximity, should compete for attentional resources in the process of updating working memory.…”

“…We focused on processing dominance when two competing types of spatial information were conveyed simultaneously through different modalities. The experiments employed a mental pathway task (Cornoldi et al, 1991;Kerr, 1993;Maezawa & Kawahara, 2021) where participants mentally maneuvered a target location according to visual or auditory directional cues and directed the target to a designated goal location. During this working memory navigation, visual and auditory directional cues appeared synchronously and indicated different directions.…”