Visualisation of different harmonisations varying in mode, consonance & dissonance

Abstract

Visual associations with auditory stimuli have been the subject of numerous studies. Colour, shape, size, and several other parameters have been linked to musical elements like timbre and pitch. In this paper we aim to examine the relationship between harmonisations with varying degrees of dissonance and visual roughness. Based on past research in which high sensory dissonance was associated with angular shapes, we argued that non-tonal and highly dissonant harmonisations will be associated with angular and rough images, while more consonant stimuli will be associated with images of low visual roughness. A fixed melody was harmonised in 7 different styles, including highly tonal, non-tonal, and random variations. Through a listening task, musically trained participants rated the stimuli in terms of enjoyment, familiarity, and matched them to images of variable roughness. The overall consonance of the stimuli was calculated using two distinct models (Wang et al., 2013; Harrison & Pearce, 2020) and a variant of the aggregate dyadic consonance index (Huron, 1994). Our results demonstrate that dissonance, as calculated by all models, was highly correlated with visual roughness, and enjoyment and familiarity followed expected patterns compared to tonal and non-tonal stimuli. In addition to sensory dissonance, however, it appears that other factors, such as the typicality of chord progressions and the sense of tonality may also influence this cross-modal interaction.