Contributions of temporal cue on the perception of speaker individuality and vocal emotion for noise-vocoded speech

Abstract

This paper investigates the importance of temporal cues in the perception of speaker individuality and vocal emotion. Experiments of speaker and vocal-emotion recognition were carried out using an analysis/synthesis method of noise-vocoded speech (NVS). The temporal resolution of NVS was controlled by varying the upper limit of modulation frequency (0, 0.5,1,2,4,8,16, 32, and 64 Hz). In addition, the role of temporal cue in the different spectral resolution condition was also investigated by varying the number of channels (4, 8, and 16). The results demonstrated that temporal resolution contributes to the recognition of both speaker and vocal emotion. Therefore, temporal cues are found to be important for the perception of not only linguistic information but also speaker individuality and vocal emotion. On the other hand, the performance of speaker recognition was less sensitive to the spectral resolution, at least in the limited set of stimuli in the present study. For vocalemotion recognition, the spectral resolution was shown to be important for recognizing only neutral, joy, and cold anger, but not sadness or hot anger. The important modulation frequency band for the perception of nonlinguistic information was suggested to be higher than that of linguistic information.

Contribution of modulation spectral features on the perception of vocal-emotion using noise-vocoded speech

“…In our previous study, in order to study the contribution of temporal modulation cues on vocal-emotion recognition, we varied the spectral and temporal resolution of noisevocoded speech stimuli presented to normal hearing listeners. The detailed method of signal processing to generate noise-vocoded speech can be found in [11]. The Fujitsu Japanese Emotional Speech Database was used.…”

“…In our previous study, we investigated the contribution of temporal modulation cues on the perception of nonlinguistic information using noise-vocoded speech [11]. The results showed that the temporal modulation cues play an important role in the perception of vocal emotion.…”

“…An analysis based on the modulation frequency analysis mechanism of the auditory system is necessary. It has been shown that auditory-based modu-lation spectral features have the potential to account for the perceptual data of vocal-emotion recognition experiments [12]. However, the specific relationship between the modulation spectral features and the perception of vocal emotion is still unknown.…”

“…In this study, the relationship between the modulation spectral features and the perceptual data of vocal-emotion recognition experiments [11] was investigated to clarify the contribution of the modulation spectral features on the perception of vocal emotion. An auditory-based modulation filterbank was used to calculate the modulation spectrograms of the temporal envelope of noise-vocoded speech.…”

“…This paper is organized as follows: Section 2 analyzes the perceptual data of vocal-emotion recognition experiments in [11]. Section 3 introduces the method for calculating the modulation spectral features from the noise-vocoded speech stimuli.…”