The organization of semantic associations between senses in language

Abstract

Distributional semantic representations were used to investigate crossmodal correspondences within language, offering a comprehensive analysis of how sensory experiences interconnect in linguistic constructs. By computing semantic proximity between words from different sensory modalities, a crossmodal semantic network was constructed, providing a general view of crossmodal correspondences in the English language. Community detection techniques were applied to unveil domains of experience where crossmodal correspondences were likely to manifest, while also considering the role of affective dimensions in shaping these domains. The study revealed the existence of an architecture of structured domains of experience in language, whereby crossmodal correspondences are deeply embedded. The present research highlights the roles of emotion and statistical associations in the organization of sensory concepts across modalities in language. The domains identified, including food, the body, the physical world and emotions/values, underscored the intricate interplay between the senses, emotion and semantic patterns. These findings align with the embodied lexicon hypothesis and the semantic coding hypothesis, emphasizing the capacity of language to capture and reflect crossmodal correspondences’ emotional and perceptual subtleties in the form of networks, while also revealing opportunities for further perceptual research on crossmodal correspondences and multisensory integration.

Crossmodal compensation: Sensory discomfort affects consumer preferences across modalities

“…Our results contribute to the literature on crossmodal associations, which has established that individuals link features of sensory stimuli across modalities (Alvarado et al, 2023;Motoki et al, 2024;Spence, 2011) (Förster et al, 2007), which would lead to assimilation effects-for example, warm temperatures increasing the desire for products associated with warmth. We find the opposite in our studies.…”

“…for different senses, many experiences are multisensory by nature, requiring the integration of sensory inputs for perception and decision-making (Driver & Spence, 2000;Senkowski et al, 2008). In line with this integrative view of sensory perception, research has demonstrated that individuals often associate sensory information across the senses (Alvarado et al, 2023;Motoki et al, 2024). In a phenomenon known as crossmodal correspondence (Spence, 2011), individuals systematically link features in one sensory modality with features in other sensory modalities.…”

“…Individuals also crossmodally link object features that share an identity or meaning-a phenomenon known as semantic congruence (Alvarado et al, 2023;Barbosa Escobar et al, 2023). For example, people associate animals with certain sounds (e.g., dogs and barking;…”