Are crossmodal correspondences relative or absolute? Sequential effects on speeded classification

Abstract

Crossmodal correspondences have often been demonstrated using congruency effects between pairs of stimuli in different sensory modalities that vary along separate dimensions. To date, however, it is still unclear the extent to which these correspondences are relative versus absolute in nature: that is, whether they result from pre-defined values that rigidly link the two dimensions or rather result from flexible values related to the previous occurrence of the crossmodal stimuli. Here, we investigated this issue in a speeded classification task featuring the correspondence between auditory pitch and visual size (e.g., congruent correspondence between high pitch/small disc and low pitch/large disc). Participants classified the size of the visual stimuli (large vs. small) while hearing concurrent high- or low-pitched task-irrelevant sounds. On some trials, visual stimuli were paired instead with "intermediate" pitch, that could be interpreted differently according to the auditory stimulus on the preceding trial (i.e., as "lower" following the presentation of a high pitch tone, but as "higher" following the presentation of a low pitch tone). Performance on sequence-congruent trials (e.g., when a small disc paired with the intermediate-pitched tone was preceded by a low pitch tone) was compared to sequence-incongruent trials (e.g., when a small disc paired with the intermediate-pitch tone was by a high-pitched tone). The results revealed faster classification responses on sequence-congruent than on sequence-incongruent trials. This demonstrates that the effect of the pitch/size correspondence is relative in nature, and subjected to trial-by-trial interpretation of the stimulus pair.

Simple and complex crossmodal correspondences involving audition

“…That said, the last few years has seen the publication of a few intriguing papers showing that individuals with synaesthesia may be more susceptible to the influences of certain crossmodal correspondences than are non-synaesthetes (e.g., see [39], see also [40]). At the same time, however, it is also worth remembering that crossmodal correspondences are often characterized as being relative [41,42] -that is, it is the larger of two circles that is matched to the lower-pitched of two sounds, rather than there being any specific match between a sound having a particular pitch and an object of a specific size. By contrast, the relation between inducer and concurrent in the case of synaesthesia proper tends to be absolute.…”

Heaviness-brightness correspondence and stimulus-response compatibility

“…Within each of the two brightness categories mapping onto different responses, heavinessbrightness congruence effect was insensitive to any gradations in the degree to which a stimulus is either brighter than the background, or darker than the background. Thus, the two brighter stimuli proved to be indistinguishable from each other in their contribution to the congruence effect, as did the two darker stimuli (see Brunetti, Indraccolo, Del Gatto, & Spence, 2018;Spence, 2019;and Walker, 2016a, for evidence for, and discussion of, the functional significance of the relative, rather than the absolute, coding of stimulus features as they contribute to correspondences).…”

Cross-modal association between vowels and colours: A cross-linguistic perspective

“…While the exact mechanism behind this frequency effect remains unknown, the influence of the individual's exposure to or experience with the linguistic items and colours on their pairings seems obvious and hard to ignore. Third, in their investigation on pitch-size correspondence, Brunetti et al (2018) demonstrated that a mediumpitch tone when preceded by a high tone (thus, relatively low) was more likely to be paired with a large disk, whereas the same medium-pitch tone when preceded by a low tone (thus, relatively high) was more likely to be paired with a small disk. Hence, it was the individual's subjective conceptualisation of "high" vs "low" that mattered, not the absolute pitch of the tone.…”

“…The difference in fundamental frequency (F0) of the stimuli is also quite large, e.g., 100-10 000 Hz in Marks (1974), either 174.6 Hz or 1046.5 Hz in Melara (1989), and 200-3417 Hz in Hubbard (1996), although de Thornley Head (2006) demonstrated that the pitch-lightness association could also be found in musical notes with a much narrower pitch range . This suggests that the pitch-lightness association is mapped relatively rather than absolutely (Brunetti et al, 2018).…”