Examining Emotion Perception Agreement in Live Music Performance

Abstract

Current music emotion recognition (MER) systems rely on emotion data averaged across listeners and over time to infer the emotion expressed by a musical piece, often neglecting time-and listener-dependent factors. These limitations can restrict the efficacy of MER systems and cause misjudgements. We present two exploratory studies on music emotion perception. First, in a live music concert setting, fifteen audience members annotated perceived emotion in the valence-arousal space over time using a mobile application. Analyses of inter-rater reliability yielded widely varying levels of agreement in the perceived emotions. A follow-up lab-based study to uncover the reasons for such variability was conducted, where twenty-one participants annotated their perceived emotions whilst viewing and listening to a video recording of the original performance and offered open-ended explanations. Thematic analysis revealed salient features and interpretations that help describe the cognitive processes underlying music emotion perception. Some of the results confirm known findings of music perception and MER studies. Novel findings highlight the importance of less frequently discussed musical attributes, such as musical structure, performer expression, and stage setting, as perceived across audio and visual modalities. Musicians are found to attribute emotion change to musical harmony, structure, and performance technique more than non-musicians. We suggest that accounting for such listener-informed music features can benefit MER in helping to address variability in emotion perception by providing reasons for listener similarities and idiosyncrasies.

TROMPA-MER: an open dataset for personalized Music Emotion Recognition

“…This task seeks to narrow the so-called semantic gap between high-level musical constructions/concepts and low-level, handcrafted representations of sound -centering it around the processing of musical content (e.g., acoustic features and lyrics). However, more recent studies have argued that user-aware music retrieval systems should include user-related factors [2][3][4][5]: user context -fluctuating characteristics from the listener, and user properties -features from the listener which are more constant. In short, user context involves collecting data regarding listening mood, uses of music, or physiological signals, while user properties include demographics, musical experience, or preference.…”

“…MER has been subject to extensive criticism given the ambiguous and subjective nature of emotions in music (see [6,[15][16][17][18]). Namely, different listeners are likely to provide diverse emotional judgments due to several factors: (1) intrinsic constructions of music (e.g., lyrics content and style), (2) sociocultural conventions (e.g., functionality of music), (3) personal differences (e.g., listener's mood, preferences, personality, and musical experience), (4) highlevel emotional evaluation mechanisms (e.g., language differences, aesthetic experience, familiarity, episodic memory, and identity confirmation), and (5) generalized confusion between the concepts of induced and perceived emotions in music. We stress the distinction between these concepts: perceived emotions are recognized by the listener through judgment/interpretation of musical properties (e.g., Western happy music is typically in major mode and has fast tempo); induced emotions are felt by the listener and involve psychophysiological responses to music (e.g., happy music might induce sadness when triggering a nostalgic memory).…”

“…Open datasets for MER have been developed mainly from benchmarking initiatives from the Audio Mood Classification task in the Music Information Retrieval Evaluation eXchange (MIREX) 5 and the Multimedia Evaluation Benchmark (MediaEval) 6 . The former released the DEAM dataset [25]: 1802 excerpts of royalty-free music rated by at least 10 MTurk 7 workers with dynamic annotations of arousal and valence.…”

TROMPA-MER: an open dataset for personalized music emotion recognition

“…This task seeks to narrow the so-called semantic gap between high-level musical constructions/concepts and low-level, handcrafted representations of sound -centering it around the processing of musical content (e.g., acoustic features and lyrics). However, more recent studies have argued that user-aware music retrieval systems should include user-related factors (Barthet et al, 2013;Schedl et al, 2013;Zangerle et al, 2021;Yang et al, 2021): user context -fluctuating characteristics from the listener, and user properties -features from the listener which are more constant. In short, user context involves collecting data regarding listening mood, uses of music, or physiological signals, while user properties include demographics, musical experience, or preference.…”

A Comparison of the Music Key Detection Approaches Utilizing Key-Profiles with a New Method Based on the Signature of Fifths

“…Major-minor tonality has long been present in Western Classical Music, providing means for shaping the mood of musical compositions. Nowadays, it plays an important role in many music-related algorithms or systems, for example music genre recognition [1][2][3], assessment of musical tension [4][5][6][7], music visualization systems [8][9][10][11], music data mining [6], computer-aided composition software [12][13][14], and determining the harmonic structure of created pieces [15,16]. It is worth mentioning that in recent years, methods implementing neural networks [17][18][19] and other machine learning approaches [20][21][22] have gained popularity in many areas of music analysis.…”