Crossmodal Correspondences in the Sounds of Chinese Instruments

Abstract

People tend to associate stimuli from different sensory modalities, a phenomenon known as crossmodal correspondences. We conducted two experiments to investigate how Chinese participants associated musical notes produced by four types of Chinese instruments (bowed strings, plucked strings, winds, and percussion instruments) with different colors, taste terms, and fabric textures. Specifically, the participants were asked to select a sound to match each color patch or taste term in Experiment 1 and to match the experience of touching each fabric in Experiment 2. The results demonstrated some associations between pitch and color, taste term, or the smoothness of fabrics. Moreover, certain types of Chinese instruments were preferentially chosen to match some of the colors, taste terms, and the texture of certain fabrics. These findings therefore provided insights about the perception of Chinese music and shed light on how to apply the multisensory features of sounds to enhance the composition, performance, and appreciation of music.

Exploring Crossmodal Associations Between Sound and the Chemical Senses: A Systematic Review Including Interactive Visualizations

“…These experiments were almost evenly split between those involving pure tones and those involving musical tones. Among musical tones, the instrumentation was an important attribute as different instruments were compared, including both Western (i.e., piano, strings, woodwind, and brass;Crisinel and Spence, 2012a, b), and non-Western (i.e., erhu, guzheng, dizi, and yunluo;Qi et al, 2020).…”

“…In 47% of the studies focusing on expectations, flavor stimuli consisted solely of terms, including taste terms (i.e., sour, sweet, bitter, salty, umami; Knoeferle et al, 2015;Wang et al, 2015Wang et al, , 2021Qi et al, 2020;Rodríguez et al, 2021) and food names (e.g., yogurt, hamburger, sandwich, chips; Motoki et al, 2022). Moreover, one experiment used audio advertisements about cream puff, lemon juice, and black coffee as chemosensory stimuli that attempted to correspond to sweet, sour, and bitter basic tastes, respectively (Motoki et al, 2019).…”

Inducing Novel Sound–Taste Correspondences via an Associative Learning Task

“…Correspondences between sound and tastes (as actual tastants and taste words) found in past literature have involved a variety of sound stimuli, including pure tones, musical tones, words, non‐words, speech sounds, music, and soundtracks (see Guedes et al., 2023, for a review), and the most prominent psychoacoustical parameter studied relates to pitch. Overall, the most consistent (but by no means perfect) correspondences found in this literature relate to associations between sweetness (and in some cases also sourness) and high‐pitch sounds and between bitterness and low‐pitch sounds (Crisinel & Spence, 2009, 2010a, 2010b; Knoeferle, Woods, Käppler, & Spence, 2015; Qi, Huang, Li, & Wan, 2020; Wang, Wang, & Spence, 2016; Q. J. Watson & Gunther, 2017). For instance, using both short chord progressions and more complex soundtracks, Knoeferle et al.…”

The taste & affect music database: Subjective rating norms for a new set of musical stimuli

“…Other studies have found similar consistencies in sound-taste mappings, particularly with pitch. Overall, high-pitched sounds were more frequently associated with either sweet and/or sour tastes (Crisinel & Spence, 2009, 2010a, 2010bKnöferle et al, 2015;Reinoso-Carvalho, Wang, de Causmaecker, et al, 2016;Velasco et al, 2014;, whereas low pitched sounds were more frequently associated with bitter tastes (Crisinel & Spence, 2009, 2010bKnöferle et al, 2015;Qi et al, 2020;Reinoso-Carvalho, Wang, de Causmaecker, et al, 2016;Velasco et al, 2014;Watson & Gunther, 2017). Associations between basic tastes and musical instruments have also been documented, for example, between sweetness and piano or bitterness and brass (Crisinel & Spence, 2010b).…”