Preschoolers’ crossmodal mappings of timbre

Abstract

Crossmodal correspondences have been of interest to researchers for nearly a century, although it is only more recently that interactions related to timbre have been examined systematically. Timbre is often described using crossmodal adjectives (e.g., bright, smooth). However, it is not clear whether these semantic conventions are primarily the result of low-level multisensory interactions or are more a product of associative learning and musical training. Do young children exhibit crossmodal correspondences involving timbre? The aim of this study was to investigate the effects of timbre, age, and sensory modality (visual and tactile) on the consistency and congruency of young children's patterns of crossmodal mappings. Preschool children (N = 69, M age = 4.51) completed a novel audio-visual and audio-tactile association task. Results indicate that children are moderately consistent in their associations; mappings are largely congruent with adult associations; and results suggest a possible developmental time course for the establishment of crossmodal correspondences between approximately ages 3 and 6. However, congruency is dependent on modality, with robust agreement in the tactile-auditory domain but a good deal more variance and a stronger developmental influence on visual-auditory associations. These results are the first to demonstrate that crossmodal correspondences are a feature of timbre perception early in development.

Roughness perception: A multisensory/crossmodal perspective

“…The results demonstrated that participants tend to match higher values of roughness to minor tonality, with some possible exceptions (e.g., Mozart, which obtained similar matching for both minor and major excerpts). Finally, Wallmark and Allen ( 2020 ) studied preschoolers’ crossmodal correspondences involving timbre (i.e., smooth vs. rough) and suggested that crossmodal timbre associations may appear early in human development (e.g., prior to substantial linguistic influence via musical training).…”

“…They found that, although presenting task-irrelevant tones (crossmodally congruent and incongruent) sped up visual responding relative to a no-sound control, there were no effects of congruency on accuracy or reaction time. Modest evidence was found that timbres increase response bias in a semantically congruent manner when participants identify visual stimuli (e.g., when a “rough” saw-tooth wave accompanies the second of two identical spatial textures, the “rough” sound increases the probability of judging the second texture as rougher), thus suggesting that rough sounds may increase judgments of roughness of the visual percepts (see also Wallmark & Allen, 2020 ).…”

The sound of timbre reigns: Auditory sequence discrimination in 3–5-year-old children

“…Children aged 4-6 years with normal hearing identified the melodies at above-chance levels (76%) using pitch and timing information alone, but were more accurate when melodies were presented in their original instrumental settings (85%), which provided rich timbre cues (as well as harmonic cues). Relatedly, Wallmark and Allen (2020) report that 3-to 5-year-olds consistently associate "rough" (sawtooth) timbres with rough tactile textures (vs. sine timbre with smooth textures). This implies perceptual sensitivity to at least some timbre properties at a young age.…”

Crossmodal associations between naturally occurring tactile and sound textures

“…The sound stimulus was also simple, pure tones. In another study examining the crossmodal mapping between sound timbre and tactile texture in young children (3–6 years old), there was an association between smoother textures for sinusoidal tones and rougher textures for sawtooth tones (both at 311 Hz) (Wallmark & Allen, 2020). In most of these studies, the study concentrated on one component of texture, like roughness, or used only pure tones.…”