Cross-modal associations between paintings and sounds: Effects of embodiment

Iosifyan, Marina; Sidoroff-Dorso, Anton V.; Wolfe, Judith

doi:10.1177/03010066221126452

Cited by 4 publications

(1 citation statement)

References 82 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In other words, artist-intended AV congruence was perceivable but did not affect subjective ratings. These results are in line with previous studies showing perceivable correspondences between art and music 13,25 , and contribute to the debate around whether such correspondences affect aesthetic experience 11,[17][18][19] .…”

Section: Discussionsupporting

confidence: 91%

The role of audiovisual congruence in perception and aesthetic appreciation of contemporary music and visual art

Fink,

Fiehn,

Wald-Fuhrmann

2023

Preprint

View full text Add to dashboard Cite

While low-level, cross-modal correspondences between vision and hearing are well-documented (e.g., Kiki-Bouba effect), it is unclear whether cross-modal correspondences are perceivable between complex, multi-dimensional stimuli, like contemporary music and art. Further, previous studies show conflicting results regarding whether audiovisual correspondence affects subjective aesthetic experience. Here, in collaboration with the Kentler International Drawing Space (NYC, USA), we use material from the Music as Image and Metaphor exhibition, consisting of music composed for each work of visual art. Our pre-registered online experiment consisted of 4 conditions: Audio, Visual, Audio-Visual-Intended (artist-intended pairing of art/music), and Audio-Visual-Random (random shuffling). Participants (N=201) were presented with 16 pieces and could click to proceed to the next piece whenever they liked. After each piece, they were asked about their subjective experience. Analyzing results by condition, we found that participants spent significantly more time with Audio, followed by Audiovisual, followed by Visual pieces; however, they felt most moved in the Audiovisual (bi-modal) conditions. The Audiovisual-Intended pieces were perceived to have greater correspondence than those in the Audiovisual-Random condition. Interestingly, though, there were no significant differences for these two conditions on any other subjective rating scale or for time spent. Collectively, these results extend our understanding of cross-modal correspondence to complex, professional, real-world abstract art and contemporary music, and call into question the use of time spent as an implicit measure of aesthetic appreciation in multi-modal conditions.

show abstract

Section: Discussionsupporting

confidence: 91%

The role of audiovisual congruence in perception and aesthetic appreciation of contemporary music and visual art

Fink,

Fiehn,

Wald-Fuhrmann

2023

Preprint

View full text Add to dashboard Cite

show abstract

Sensory Profiles and Technological Deficits: Classification, Relevance, and Gaps

Hutson,

Harper-Nichols

2024

Sensemaking and Neuroaesthetics

View full text Add to dashboard Cite

The role of audiovisual congruence in aesthetic appreciation of contemporary music and visual art

Fink,

Fiehn,

Wald-Fuhrmann

2024

Sci Rep

View full text Add to dashboard Cite

Does congruence between auditory and visual modalities affect aesthetic experience? While cross-modal correspondences between vision and hearing are well-documented, previous studies show conflicting results regarding whether audiovisual correspondence affects subjective aesthetic experience. Here, in collaboration with the Kentler International Drawing Space (NYC, USA), we depart from previous research by using music specifically composed to pair with visual art in the professionally-curated Music as Image and Metaphor exhibition. Our pre-registered online experiment consisted of 4 conditions: Audio, Visual, Audio-Visual-Intended (artist-intended pairing of art/music), and Audio-Visual-Random (random shuffling). Participants (N = 201) were presented with 16 pieces and could click to proceed to the next piece whenever they liked. We used time spent as an implicit index of aesthetic interest. Additionally, after each piece, participants were asked about their subjective experience (e.g., feeling moved). We found that participants spent significantly more time with Audio, followed by Audiovisual, followed by Visual pieces; however, they felt most moved in the Audiovisual (bi-modal) conditions. Ratings of audiovisual correspondence were significantly higher for the Audiovisual-Intended compared to Audiovisual-Random condition; interestingly, though, there were no significant differences between intended and random conditions on any other subjective rating scale, or for time spent. Collectively, these results call into question the relationship between cross-modal correspondence and aesthetic appreciation. Additionally, the results complicate the use of time spent as an implicit measure of aesthetic experience.

show abstract

Cross-modal associations between paintings and sounds: Effects of embodiment

Cited by 4 publications

References 82 publications

The role of audiovisual congruence in perception and aesthetic appreciation of contemporary music and visual art

The role of audiovisual congruence in perception and aesthetic appreciation of contemporary music and visual art

Sensory Profiles and Technological Deficits: Classification, Relevance, and Gaps

The role of audiovisual congruence in aesthetic appreciation of contemporary music and visual art

Contact Info

Product

Resources

About