Neurocomputational mechanisms underlying cross-modal associations and their influence on perceptual decisions

Bolam, Joshua; Boyle, Stephanie Claire; Ince, Robin A. A.; Delis, Ioannis

doi:10.1016/j.neuroimage.2021.118841

Cited by 6 publications

(7 citation statements)

References 136 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nevertheless, it could also be argued that by establishing an intermediate link, the association could be more robust and last longer, as a correlational/statistical path has already been set for the correspondence to develop via a top‐down approach. As past literature has shown, both bottom‐up and top‐down approaches play a role in the formation of crossmodal correspondences (Bolam, Boyle, Ince, & Delis, 2022; Getz & Kubovy, 2018; Peiffer‐Smadja & Cohen, 2019). That being said, to the best of our knowledge, the present study is the first one to investigate the lifespan of experimentally induced crossmodal correspondences.…”

Section: Discussionmentioning

confidence: 99%

Inducing Novel Sound–Taste Correspondences via an Associative Learning Task

Barbosa Escobar,

Wang

2024

Cognitive Science

View full text Add to dashboard Cite

The interest in crossmodal correspondences, including those involving sounds and involving tastes, has experienced rapid growth in recent years. However, the mechanisms underlying these correspondences are not well understood. In the present study (N = 302), we used an associative learning paradigm, based on previous literature using simple sounds with no consensual taste associations (i.e., square and triangle wave sounds at 200 Hz) and taste words (i.e., sweet and bitter), to test the influence of two potential mechanisms in establishing sound–taste correspondences and investigate whether either learning mechanism could give rise to new and long‐lasting associations. Specifically, we examined an emotional mediation account (i.e., using sad and happy emoji facial expressions) and a transitive path (i.e., sound‐taste correspondence being mediated by color, using red and black colored squares). The results revealed that the associative learning paradigm mapping the triangle wave tone with a happy emoji facial expression induced a novel crossmodal correspondence between this sound and the word sweet. Importantly, we found that this novel association was still present two months after the experimental learning paradigm. None of the other mappings, emotional or transitive, gave rise to any significant associations between sound and taste. These findings provide evidence that new crossmodal correspondences between sounds and tastes can be created by leveraging the affective connection between both dimensions, helping elucidate the mechanisms underlying these associations. Moreover, these findings reveal that these associations can last for several weeks after the experimental session through which they were induced.

show abstract

Section: Discussionmentioning

confidence: 99%

Inducing Novel Sound–Taste Correspondences via an Associative Learning Task

Barbosa Escobar,

Wang

2024

Cognitive Science

View full text Add to dashboard Cite

show abstract

“…Therefore, this study extends the theory of musical concordance to crossmodal research. Most previous research has focused more on the crossmodal association between the fixed pitch and figure (Bolam et al, 2022; Bonetti & Costa, 2018; Gallace & Spence, 2006). However, the world we live in is constantly evolving.…”

Section: Discussionmentioning

confidence: 99%

“…Participants' response time was significantly faster under the congruent, rather than the incongruent condition. This discovery has been confirmed many times in recent years (Bolam et al, 2022;Bonetti & Costa, 2018), and the pitch-size correspondence is a sensory processing feature shared by other species rather than unique to humans (Korzeniowska et al, 2022). In theory, this crossmodal correspondence between sound and figure can be explained by Bayesian "coupling priors," namely, perception is influenced by expectations from the environment (Klaffehn et al, 2021).…”

mentioning

confidence: 80%

See 1 more Smart Citation

Dynamic audio-visual correspondence in musicians and non-musicians

Guo,

Qu,

Liu

et al. 2023

Psychology of Music

View full text Add to dashboard Cite

Most previous audio-visual crossmodal correspondence studies focused on static visual or auditory stimuli. Moreover, some researchers have found that music training can effectively improve the interaction between auditory and visual channels by enhancing neural plasticity. This study focused on whether crossmodal correspondence occurs when people face dynamic visual and auditory stimuli, and whether it is affected by musical training. Participants were asked to judge different changes in pitch (rise and drop) by showing them different patterns of visual circle motion (enlarged, reduced, and unchanged). The results revealed that the audio-visual congruent combinations (the pitch rise when the circle was enlarged and the pitch drop when the circle was reduced) significantly shortened participants’ response times, whereas the audio-visual neutral combinations (the unchanged shape) had the highest accuracy. Participants with musical training were faster than participants without musical training to judge pitch changes in conditions where the association between pitch and shape size was incongruent. These findings provide empirical evidence for dynamic audio-visual crossmodal correspondence and shed light on the bright prospect of using congruent audio-visual stimuli in animation.

show abstract

“…This analysis was crucial because the results of Experiment 2 did not clearly speak to why auditory–visual intensity correspondence does not manifest in both classification tasks. To our knowledge, none of these three analysis methods have previously been applied to study crossmodal correspondences in speeded classification tasks (but see e.g., Bolam et al, 2022; Parise & Spence, 2012, for distributional and diffusion model analyses of crossmodal implicit association effects).…”

Section: Conflict-task Modeling Of Crossmodal Intensity Correspondencementioning

confidence: 99%