Contribution of Common Modulation Spectral Features to Vocal-Emotion Recognition of Noise-Vocoded Speech in Noisy Reverberant Environments

Abstract

In one study on vocal emotion recognition using noise-vocoded speech (NVS), the high similarities between modulation spectral features (MSFs) and the results of vocal-emotion-recognition experiments indicated that MSFs contribute to vocal emotion recognition in a clean environment (with no noise and no reverberation). Other studies also clarified that vocal emotion recognition using NVS is not affected by noisy reverberant environments (signal-to-noise ratio is greater than 10 dB and reverberation time is less than 1.0 s). However, the contribution of MSFs to vocal emotion recognition in noisy reverberant environments is still unclear. We aimed to clarify whether MSFs can be used to explain the vocal-emotion-recognition results in noisy reverberant environments. We analyzed the results of vocal-emotion-recognition experiments and used an auditory-based modulation filterbank to calculate the modulation spectrograms of NVS. We then extracted ten MSFs as higher-order statistics of modulation spectrograms. As shown from the relationship between MSFs and vocal-emotion-recognition results, except for extremely high noisy reverberant environments, there were high similarities between MSFs and the vocal emotion recognition results in noisy reverberant environments, which indicates that MSFs can be used to explain such results in noisy reverberant environments. We also found that there are two common MSFs (MSKTk (modulation spectral kurtosis) and MSTLk (modulation spectral tilt)) that contribute to vocal emotion recognition in all daily environments.

Contributions of Temporal Modulation Cues in Temporal Amplitude Envelope of Speech to Urgency Perception

“…Our previous experiments in which we systematically varied the number of channels in the NVS scheme and the upper limitation of the modulation frequency components [11,12] demonstrated that temporal modulation cues play a key role in the recognition of both vocal emotion and speaker individuality, particularly at modulation frequencies lower than 8 Hz. Our research has also demonstrated that these features are robust to noise and reverberation [13,14]. However, we have still not clarified the influence of temporal modulation cues in non-linguistic information such as that used in emergency situations.…”

“…First, we reduced the effect of the average intensity by normalizing the active sp levels of all speech signals to −26 dBov with a P.56 speech voltmeter [14]. We then plemented band-pass filters (BPF) (essentially functioning as a band-pass filterbank divide the signals into several frequency bands.…”

“…where f refers to the center frequency in Hz. We define the center frequencies of the b pass filterbank from 3 to 35 Cam with the bandwidth set to 2 × ERBN, resulting First, we reduced the effect of the average intensity by normalizing the active speech levels of all speech signals to −26 dBov with a P.56 speech voltmeter [14]. We then implemented band-pass filters (BPF) (essentially functioning as a band-pass filterbank) to divide the signals into several frequency bands.…”