A Discriminative Feature Representation Method Based on Cascaded Attention Network With Adversarial Strategy for Speech Emotion Recognition

Liu, Yang; Sun, Haoqin; Guan, Wenbo; Xia, Yuqi; Li, Yongwei; Unoki, Masashi; Zhen, Zhao

doi:10.1109/taslp.2023.3245401

Cited by 10 publications

(1 citation statement)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We chose it for the following reasons. Firstly, this database is often used in research on emotional speech [22,23] and is widely available. Secondly, as presented by the authors of [24], this database enables the highest efficiency of classification (88.47% [25]) compared to other databases, such as RAVDESS (87.5% [15,26]) or IEMOCAP (75.60% [27,28]).…”

Section: Audio Datamentioning

confidence: 99%

Interpreting Convolutional Layers in DNN Model Based on Time–Frequency Representation of Emotional Speech

Smietanka,

Maka

2023

Journal of Artificial Intelligence and Soft Computing Research

View full text Add to dashboard Cite

The paper describes the relations of speech signal representation in the layers of the convolutional neural network. Using activation maps determined by the Grad-CAM algorithm, energy distribution in the time–frequency space and their relationship with prosodic properties of the considered emotional utterances have been analysed. After preliminary experiments with the expressive speech classification task, we have selected the CQT-96 time–frequency representation. Also, we have used a custom CNN architecture with three convolutional layers in the main experimental phase of the study. Based on the performed analysis, we show the relationship between activation levels and changes in the voiced parts of the fundamental frequency trajectories. As a result, the relationships between the individual activation maps, energy distribution, and fundamental frequency trajectories for six emotional states were described. The results show that the convolutional neural network in the learning process uses similar fragments from time–frequency representation, which are also related to the prosodic properties of emotional speech utterances. We also analysed the relations of the obtained activation maps with time-domain envelopes. It allowed observing the importance of the speech signals energy in classifying individual emotional states. Finally, we compared the energy distribution of the CQT representation in relation to the regions’ energy overlapping with masks of individual emotional states. In the result, we obtained information on the variability of energy distributions in the selected signal representation speech for particular emotions.

show abstract