Comparison of Japanese expressive speech perception by Japanese and Taiwanese listeners

Abstract

Language is an important tool in speech communication. Even without the understanding of one language, we can still judge the expressive content of a voice, such as happiness or sadness. However, sometimes misunderstanding of emotional communication occurs. It is not clear what the common/different features are that help or hinder people with different culture/native-languages background in making judgments about the expressivity of speech. In order to explore this question, we focus on Japanese and Taiwanese listeners perception of Japanese expressive speech utterances. We used the perceptual model proposed by [Huang and Akagi, InterSpeech 2005; 2007], which involves a concept called "semantic primitives"-- adjectives for describing voice perception. This concept simplifies and clarifies the discussion of common/different features in terms of acoustic cues and expressive speech perception categories. Using this model, we found some common/different features in the perception of expressive speech. Specifically, our results suggest that there may be primary and secondary semantic primitives associated with acoustic speech characteristics which are involved in the perception of expressive speech, and that people from different cultures/native-language background tend to use the same primary semantic primitives in perceiving expressive speech but different secondary ones.

Cross-Cultural Emotion Recognition and In-Group Advantage in Vocal Expression: A Meta-Analysis

“…The majority of studies also used a forced-choice method wherein participants were asked to choose their response from a list of predefined response alternatives. However, a couple of studies that used rating scales or free responses were included because they also provided data on the proportion of accurate responses (Abelin & Allwood, 2000; Gendron, Roberson, van der Vyver, & Barrett, 2014; Huang, Erickson, & Akagi, 2008). The accuracy criterion excluded studies that investigated emotion recognition using dimensional ratings (e.g., activation, valence, emotion intensity, appraisal dimensions)—which do not provide measures of accuracy (e.g., Koeda et al, 2013; Nordström, Laukka, Thingujam, Schubert, & Elfenbein, 2017; Pfitzinger, Amir, Mixdorff, & Bösel, 2011).…”

A Three-Layer Emotion Perception Model for Valence and Arousal-Based Detection from Multilingual Speech

“…Low arousal and negative valence speech easily make an impression on listeners with dark and heavy feelings, but high arousal and positive valence speech is oftentimes uttered in a bright and well-modulated way. The set of semantic primitives derived from [4] that we examined and used in the three-layer model for describing emotional speech was: bright, dark, high, low, strong, weak, calm, unstable, well-modulated, monotonous, heavy, clear, noisy, quiet, sharp, fast, and slow. To construct the three-layer model, the three emotional corpora were first evaluated in terms of each semantic primitive through human listening tests.…”

“…The purpose is to create a natural human-machine interaction in a real-world-context [1]. These days, psychology has proven that the human ability to perceive emotions is cross-lingual with no utterance required in his/her native or foreign language [2][3][4] . Despite great progress made in SER, natural human-behavior interaction is still an obstacle, on the grounds that there is a strong dependence on the language being spoken, i.e.…”

Toward relaying an affective Speech-to-Speech translator: Cross-language perception of emotional state represented by emotion dimensions

“…People can still judge the expressive content of a voice for one language, such as emotional states, even without the understanding of that language [6]. Several studies have indeed shown evidence for certain universal attributes for speech [7], not only among individuals of the same culture, but also across cultures.…”