Roughness perception: A multisensory/crossmodal perspective

Abstract

Roughness is a perceptual attribute typically associated with certain stimuli that are presented in one of the spatial senses. In auditory research, the term is typically used to describe the harsh effects that are induced by particular sound qualities (i.e., dissonance) and human/animal vocalizations (e.g., screams, distress cries). In the tactile domain, roughness is a crucial factor determining the perceptual features of a surface. The same feature can also be ascertained visually, by means of the extraction of pattern features that determine the haptic quality of surfaces, such as grain size and density. By contrast, the term roughness has rarely been applied to the description of those stimuli perceived via the chemical senses. In this review, we take a critical look at the putative meaning(s) of the term roughness, when used in both unisensory and multisensory contexts, in an attempt to answer two key questions: (1) Is the use of the term ‘roughness’ the same in each modality when considered individually? and (2) Do crossmodal correspondences involving roughness match distinct perceptual features or (at least on certain occasions) do they merely pick-up on an amodal property? We start by examining the use of the term in the auditory domain. Next, we summarize the ways in which the term roughness has been used in the literature on tactile and visual perception, and in the domain of olfaction and gustation. Then, we move on to the crossmodal context, reviewing the literature on the perception of roughness in the audiovisual, audiotactile, and auditory-gustatory/olfactory domains. Finally, we highlight some limitations of the reviewed literature and we outline a number of key directions for future empirical research in roughness perception.

Explaining Visual Shape–Taste Crossmodal Correspondences

“…The latter is a rare phenomenon, and defined in terms of idiosyncratic connections between inducer and concurrent (Grossenbacher and Lovelace, 2001). Instead, researchers have sought an explanation in terms of the increasingly popular crossmodal correspondences (Di Stefano and Spence, 2022;Spence, 2011a). Confusing matters somewhat, though, synaesthetes are presumably likely to experience many of the same crossmodal correspondences as non-synaesthetes, hence perhaps explaining the angularity-acidity connection picked-up in the responses of the taste-shape synaesthete in Cytowic and Woods ' (1982) early study.…”

“…So, for example, participants have sometimes felt (and/or seen) a textured plate or cup, or else been asked to rub a swatch of silk or a piece of sandpaper, say, prior to and/or while tasting a particular food or drink, and rating the latter's taste. In such cases, however, it is possible to imagine a more or less direct mapping, or translation, of seen/felt texture to the matching mouthfeel (though see Di Stefano and Spence, 2022). Texture descriptors such as 'smooth' are also commonly applied to taste (see Burke, 2009, p. 130), though, in such cases, it is easy to imagine that what is being described is the literal feel of a given foodstuff in the oral cavity rather than a more metaphorical description (Note 4).…”

Sensory translation between audition and vision

“…The composer Raff also reported that he perceived the colour of the sound of the trumpet to be scarlet (other people apparently report it to be bright red; Ortmann, 1933 ), while, for Kandinsky, the sound of the tuba was also red (see also Anikin & Johansson, 2019 ; Donnell-Kotrozo, 1978 ; Ginsberg, 1923 , p. 589; Menouti et al, 2015 ; Reuter et al, 2018b ). Other contemporary researchers, meanwhile, have chosen to study the correspondences between timbre and shape (see Adeli et al, 2014 ; Gurman et al, 2021 ) or harmony (namely, consonance and dissonance) and visual roughness (Giannos et al, 2021 ; see Di Stefano & Spence, 2022 , for a review on multisensory roughness, and Di Stefano et al, 2022b for a review on consonance and dissonance) (Table 3 ).…”

Tactile perception of auditory roughness

“…In a recent review on multimodal roughness perception, Di Stefano & Spence (2022) made a clear distinction between auditory roughness and surface roughness: auditory roughness is a temporally based perceptual property, that is experienced through hearing, while surface roughness is spatiotemporal one, related to textures, that is assessed by touch (and vision). However, whether the sensation of auditory roughness as historically investigated could also be experienced through touch is still unclear.…”