Neural basis of sound-symbolic pseudoword-shape correspondences

Barany, Deborah A.; Lacey, Simon; Matthews, Kaitlyn L.; Nygaard, Lynne C.; Sathian, K.

doi:10.1016/j.neuropsychologia.2023.108657

Cited by 2 publications

(3 citation statements)

References 84 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sound symbolism also seems to be related to the neural basis of language. In a recent functional magnetic resonance imaging (fMRI) study, a multivoxel pattern analysis (MVPA) indicated that activity in language-associated areas such as the left supramarginal gyrus and Broca’s area in the left inferior frontal gyrus could distinguish rounded/pointed stimuli more accurately for sound symbolically matched pseudoword-shape pairs (e.g., /molo/ + rounded shape) than for mismatched pairs (e.g., /molo/ + pointed shape) ( Barany et al, 2023 ).…”

Section: Is Lexical Information Formed Independent Of Multisensory Pr...mentioning

confidence: 99%

“…There is also evidence that sound symbolism may involve multisensory processing. For example, sound symbolically matched audiovisual pseudoword-shape pairs produce more activation in auditory areas than unmatched pairs ( Barany et al, 2023 ). In visual areas, the activation difference between mismatched and matched pseudoword-shape pairs correlates with behavioral measures of implicit pseudoword-shape associations ( Peiffer-Smadja and Cohen, 2019 ).…”

Section: Is Lexical Information Formed Independent Of Multisensory Pr...mentioning

confidence: 99%

“…In visual areas, the activation difference between mismatched and matched pseudoword-shape pairs correlates with behavioral measures of implicit pseudoword-shape associations ( Peiffer-Smadja and Cohen, 2019 ). Likewise, MVPA indicates that activity in early visual areas more accurately distinguishes rounded and pointed stimuli that are part of sound symbolically matched, as opposed to mismatched, pseudoword-shape pairs ( Barany et al, 2023 ). While the exact nature of the neural computations underlying sound symbolism are still not understood, these findings indicate that sound symbolism may, at least partly, involve multisensory processing.…”

Section: Is Lexical Information Formed Independent Of Multisensory Pr...mentioning

confidence: 99%

See 2 more Smart Citations

Multisensory and lexical information in speech perception

Dorsi,

Lacey,

Sathian

2024

Front. Hum. Neurosci.

Self Cite

View full text Add to dashboard Cite

Both multisensory and lexical information are known to influence the perception of speech. However, an open question remains: is either source more fundamental to perceiving speech? In this perspective, we review the literature and argue that multisensory information plays a more fundamental role in speech perception than lexical information. Three sets of findings support this conclusion: first, reaction times and electroencephalographic signal latencies indicate that the effects of multisensory information on speech processing seem to occur earlier than the effects of lexical information. Second, non-auditory sensory input influences the perception of features that differentiate phonetic categories; thus, multisensory information determines what lexical information is ultimately processed. Finally, there is evidence that multisensory information helps form some lexical information as part of a phenomenon known as sound symbolism. These findings support a framework of speech perception that, while acknowledging the influential roles of both multisensory and lexical information, holds that multisensory information is more fundamental to the process.

show abstract

Section: Is Lexical Information Formed Independent Of Multisensory Pr...mentioning

confidence: 99%

Section: Is Lexical Information Formed Independent Of Multisensory Pr...mentioning

confidence: 99%

Section: Is Lexical Information Formed Independent Of Multisensory Pr...mentioning

confidence: 99%

See 1 more Smart Citation

Multisensory and lexical information in speech perception

Dorsi,

Lacey,

Sathian

2024

Front. Hum. Neurosci.

Self Cite

View full text Add to dashboard Cite

show abstract

Crossmodal correspondences between visual and speech angularity and tactile jaggedness of response key

Suzuki,

Ueno,

Nakahashi

et al. 2024

Sci Rep

View full text Add to dashboard Cite

Several studies reported various crossmodal correspondences related to tactile features. These previous studies have investigated tactile-related correspondences through explicit matching or subjective evaluation tasks, which required participants to recognize relationships between tactile and other sensory features or rate tactile materials on scales with adjective labels related to visual or auditory features. However, these tasks are prone to occur the experimenter-expectancy effects and arbitrary categorization of tactile materials by the labels, making it difficult to assess implicit and non-arbitrary aspects of crossmodal correspondences. To address this, we used a speeded classification task to examine whether the angularity/roundedness of visual and auditory stimuli correspond to tactile jaggedness/fluffiness. Participants distinguished between angularity or roundedness (Experiment 1: visual shapes; Experiment 2: speech sounds) by pressing right- or left-positioned response keys with task-irrelevant jaggedness or fluffiness without prior instruction on which key represented jaggedness/fluffiness. Results showed faster keypresses for jagged/fluffy responses to angular/rounded stimuli, suggesting an implicit correspondence between these sensory features except for the experimenter-expectancy effects and the influence of the labels. Unlike previous studies that examined the correspondence with simple tactile features (e.g., weight, size), our findings suggest that even complex tactile-quality features, such as jaggedness/fluffiness, implicitly correspond to visual and auditory angularity/roundedness.

show abstract

Neural basis of sound-symbolic pseudoword-shape correspondences

Cited by 2 publications

References 84 publications

Multisensory and lexical information in speech perception

Multisensory and lexical information in speech perception

Crossmodal correspondences between visual and speech angularity and tactile jaggedness of response key

Contact Info

Product

Resources

About