Multimodal Classification of Emotions in Latin Music

Catharin, Leonardo Gabiato; Ribeiro, Rafael P.; Silla, Carlos N.; Costa, Yandre M. G.; Feltrim, Valéria Delisandra

doi:10.1109/ism.2020.00038

Cited by 3 publications

(1 citation statement)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As AI and deep learning are both developing fast these days, significant progresses have been made in fields of computer vision and natural language processing [1][2][3][4][5], which has created a background for the emergence of MAC for combined processing of texts and images [6][7][8][9][10][11][12]. Conventional teaching evaluation emphasizes students' knowledge mastery over their affections [13][14][15], however, the state of affection has a very important influence on students' learning process and outcome.…”

Section: Introductionmentioning

confidence: 99%

Multi-Modal Affective Computing: An Application in Teaching Evaluation Based on Combined Processing of Texts and Images

Wang¹,

Zhai²,

Xu³

et al. 2023

View full text Add to dashboard Cite

Conventional teaching evaluation emphasizes students' knowledge mastery over their affections. Multi-modal Affective Computing (MAC) can analyze versatile information of students in the classroom, including their facial expressions, gestures, and text feedback, in a comprehensive way, thereby helping teachers discover problems with students' affections in a timely manner, so that they could adjust the teaching methods and strategies accordingly. However, the available MAC technology might make unstable or wrong judgement when dealing with complex affective expressions, then the inaccurate evaluation results of students' affection state might adversely affect the teaching evaluation results. To tackle these issues, this study innovatively applied MAC in teaching evaluation based on combined processing of texts and images. The input texts were divided into two parts: main body and the hash tag, which were subjected to feature extraction respectively. The image features were extracted from two angles: object and scene, since the two angles can give image information of different levels. The MAC model was divided into modal sharing tasks and modal private tasks to attain better adaptability in case of new teaching evaluation scenarios. The effectiveness of the proposed method was verified by experimental results.

show abstract

Section: Introductionmentioning

confidence: 99%

Multi-Modal Affective Computing: An Application in Teaching Evaluation Based on Combined Processing of Texts and Images

Wang¹,

Zhai²,

Xu³

et al. 2023

View full text Add to dashboard Cite

show abstract

Analysis of Traditional Quyi Emotion Classification Algorithm Driven by Deep Belief Network

2022

Application of Intelligent Systems in Multi-Modal Information Analytics

View full text Add to dashboard Cite

MMD-MII Model: A Multilayered Analysis and Multimodal Integration Interaction Approach Revolutionizing Music Emotion Classification

Wang,

Sharifi,

Gadekallu

et al. 2024

Int J Comput Intell Syst

View full text Add to dashboard Cite

Music plays a vital role in human culture and society, serving as a universal form of expression. However, accurately classifying music emotions remains challenging due to the intricate nature of emotional expressions in music and the integration of diverse data sources. To address these challenges, we propose the Multilayered Music Decomposition and Multimodal Integration Interaction (MMD-MII) model. This model employs cross-processing to facilitate interaction between audio and lyrics, ensuring coherence in emotional representation. Additionally, we introduce a hierarchical framework based on the music theory, focusing on the main and chorus sections, with the chorus processed separately to extract precise emotional representations. Experimental results on the DEAM and FMA datasets demonstrate the effectiveness of the MMD-MII model, achieving accuracies of 49.68% and 49.54% respectively. Compared with the existing methods, our model outperforms in accuracy and F1 scores, offering promising implications for music recommendation systems, healthcare, psychology, and advertising, where accurate emotional analysis is essential.

show abstract

Multimodal Classification of Emotions in Latin Music

Cited by 3 publications

References 15 publications

Multi-Modal Affective Computing: An Application in Teaching Evaluation Based on Combined Processing of Texts and Images

Multi-Modal Affective Computing: An Application in Teaching Evaluation Based on Combined Processing of Texts and Images

Analysis of Traditional Quyi Emotion Classification Algorithm Driven by Deep Belief Network

MMD-MII Model: A Multilayered Analysis and Multimodal Integration Interaction Approach Revolutionizing Music Emotion Classification

Contact Info

Product

Resources

About