Automatic Identification of Emotional Information in Spanish TV Debates and Human–Machine Interactions

Velasco, Mikel de; Justo, Raquel; Torres, M. Inés

doi:10.3390/app12041902

Cited by 3 publications

(2 citation statements)

References 69 publications

(73 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For future work it would be interesting to explore other classification architectures and label more data to improve the results and make it possible to learn representations for more classes. For example, it would be interesting to have more data of the Enthusiastic class, since, as seen in Figure 1 and in [12], it is quite distinguishable from other emotions in our corpus.…”

Section: Discussionmentioning

confidence: 99%

“…In contrast with previous research [12] in which audio segments between 2 and 5 seconds were considered, in this work we used the full audio of each speaker intervention without slicing it since we considered this could be a more representative unit for emotional recognition.The audio files in which speakers could not be told apart and the ones that were not related to the debates (music, ads, etc.) were removed from the corpus.…”

Section: Spanishmentioning

confidence: 99%

See 1 more Smart Citation

Speech emotion recognition in Spanish TV Debates

Zubiaga¹,

Justo²,

Torres³

et al. 2022

IberSPEECH 2022

View full text Add to dashboard Cite

Emotion recognition from speech is an active field of study that can help build more natural human-machine interaction systems. Even though the advancement of deep learning technology has brought improvements in this task, it is still a very challenging field. For instance, when considering real life scenarios, things such as tendency toward neutrality or the ambiguous definition of emotion can make labeling a difficult task causing the data-set to be severally imbalanced and not very representative.In this work we considered a real life scenario to carry out a series of emotion classification experiments. Specifically, we worked with a labeled corpus consisting of a set of audios from Spanish TV debates and their respective transcriptions. First, an analysis of the emotional information within the corpus was conducted. Then different data representations were analyzed as to choose the best one for our task; Spectrograms and UniSpeech-SAT were used for audio representation and Dis-tilBERT for text representation. As a final step, Multimodal Machine Learning was used with the aim of improving the obtained classification results by combining acoustic and textual information.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Spanishmentioning

confidence: 99%

Speech emotion recognition in Spanish TV Debates

Zubiaga¹,

Justo²,

Torres³

et al. 2022

IberSPEECH 2022

View full text Add to dashboard Cite

show abstract

Analysis of Deep Learning-Based Decision-Making in an Emotional Spontaneous Speech Task

et al. 2023

View full text Add to dashboard Cite

In this work, we present an approach to understand the computational methods and decision-making involved in the identification of emotions in spontaneous speech. The selected task consists of Spanish TV debates, which entail a high level of complexity as well as additional subjectivity in the human perception-based annotation procedure. A simple convolutional neural model is proposed, and its behaviour is analysed to explain its decision-making. The proposed model slightly outperforms commonly used CNN architectures such as VGG16, while being much lighter. Internal layer-by-layer transformations of the input spectrogram are visualised and analysed. Finally, a class model visualisation is proposed as a simple interpretation approach whose usefulness is assessed in the work.

show abstract

Dialogue Management and Language Generation for a Robust Conversational Virtual Coach: Validation and User Study

Vázquez

Zorrilla

Olaso

et al. 2023

Sensors

Self Cite

View full text Add to dashboard Cite

Designing human–machine interactive systems requires cooperation between different disciplines is required. In this work, we present a Dialogue Manager and a Language Generator that are the core modules of a Voice-based Spoken Dialogue System (SDS) capable of carrying out challenging, long and complex coaching conversations. We also develop an efficient integration procedure of the whole system that will act as an intelligent and robust Virtual Coach. The coaching task significantly differs from the classical applications of SDSs, resulting in a much higher degree of complexity and difficulty. The Virtual Coach has been successfully tested and validated in a user study with independent elderly, in three different countries with three different languages and cultures: Spain, France and Norway.

show abstract

Automatic Identification of Emotional Information in Spanish TV Debates and Human–Machine Interactions

Cited by 3 publications

References 69 publications

Speech emotion recognition in Spanish TV Debates

Speech emotion recognition in Spanish TV Debates

Analysis of Deep Learning-Based Decision-Making in an Emotional Spontaneous Speech Task

Dialogue Management and Language Generation for a Robust Conversational Virtual Coach: Validation and User Study

Contact Info

Product

Resources

About