No abstract

Several pairs of Japanese adjective words pertaining to material's properties, such as roughness and hardness, have been used in Japanese studies to quantitatively evaluate variations in tactile sensations. This method asks observers to analyze their perceptual experiences one by one. An alternative notion is that human perceptual recognition is performed as a whole rather than by using fragmented factors. Based on this notion, we propose a system that can automatically estimate multidimensional ratings of touch from a single sound-symbolic word that has been spontaneously and intuitively expressed by a user. When a user inputs a sound-symbolic word into the system, the system refers to a database of phonemes and their auditory impressions, and calculates ratings in terms of 26 pairs of fundamental scales of touch. The estimated ratings of sound-symbolic words enable us to visualize a tactile perceptual space. Our study provides an alternative method for estimating the fine quality of tactile sensations.

Unlike the assumption of modern linguistics, there is non-arbitrary association between sound and meaning in sound symbolic words. Neuroimaging studies have suggested the unique contribution of the superior temporal sulcus to the processing of sound symbolism. However, because these findings are limited to the mapping between sound symbolism and visually presented objects, the processing of sound symbolic information may also involve the sensory-modality dependent mechanisms. Here, we conducted a functional magnetic resonance imaging experiment to test whether the brain regions engaged in the tactile processing of object properties are also involved in mapping sound symbolic information with tactually perceived object properties. Thirty-two healthy subjects conducted a matching task in which they judged the congruency between softness perceived by touch and softness associated with sound symbolic words. Congruency effect was observed in the orbitofrontal cortex, inferior frontal gyrus, insula, medial superior frontal gyrus, cingulate gyrus, and cerebellum. This effect in the insula and medial superior frontal gyri was overlapped with softness-related activity that was separately measured in the same subjects in the tactile experiment. These results indicate that the insula and medial superior frontal gyrus play a role in processing sound symbolic information and relating it to the tactile softness information.

In medical interviews, Japanese patients often use onomatopoeia, such as ‘zuki-zuki’ and ‘chiku-chiku,’ to express pain symptoms and medical conditions. However, onomatopoeia shows cross-linguistic variation, and thus Japanese onomatopoeia cannot be used effectively to express pain symptoms in medical interviews with foreign doctors who do not speak Japanese. In this study, we developed a system that supports communication between Japanese patients and foreign doctors by putting an onomatopoeia evaluation system to medical use. Our system estimates the quality of pain and other medical conditions based on the sound symbolic meanings expressed by certain onomatopoetic expressions. The relationships between the sound symbolic properties and rating scales were obtained through psychological experiments in which 120 participants evaluated the mental images of 354 Japanese onomatopoeia terms used to express pain symptoms and medical conditions against 35 semantic differential (SD) scales such as “sharp–dull,” “strong–weak,” and “momentary–continuous.” Our system accepts any Japanese onomatopoetic expression input by users and can also respond to any novel onomatopoetic expression. If the rating scales were translated into various languages, foreign doctors all over the world would be able to understand the meaning of Japanese onomatopoeia.

No abstract