EMuJoy: Software for continuous measurement of perceived emotions in music

Abstract

Since Kate Hevner's early investigations on the perception of emotions while listening to music (Hevner, 1936), there have been many different approaches to the measurement of emotions. For example, Gabrielsson and Lindström Wik (2003) investigated musical expression by describing the verbal reports given by subjects. Most researchers have used distinct adjective scales for the rating of perceived or expressed emotions. Schlosberg (1954) found that such scales can be mapped onto two or three dimensions. Using these methods, self-reported data were collected on distinct and mostly nonequidistant points in time, which were chosen intuitively by the researcher.The experience of emotion in music and films unfolds over time: This has led to increasing interest in the continuous measurement of perceived emotions, made possible by technological developments since the 1990s. Subjects' responses can now be recorded in real time and synchronized to the stimuli. Schubert (1996Schubert ( , 2001Schubert ( /2002Schubert ( , 2004aSchubert ( , 2004b was one of the first researchers to develop software that focuses on the perception of the temporal dynamics of emotion.However, up to now, methods for the recording of continuous responses were based on researcher-developed software solutions, and there was no agreement on the technical means, interfaces, or methods to use. The main aim of this contribution to the ongoing discussion is to propose standardized methods for the continuous measurement of self-reported emotions. The authors have designed new software, EMuJoy, for this purpose. It is freeware and can be distributed and used for research.Before describing our integrated software solution, we will address four questions: (1) the dimensionality of the emotion space, (2) technical aspects of data recording, (3) construction of the interface, and (4) the use of multiple stimulus modalities. Dimensionality of the Emotion SpaceUtilizing similarity ratings of affect-descriptive terms, Russell (1980) demonstrated in his circumplex model of affects that such terms can be mapped onto a two-dimensional space. He analyzed the similarity matrix, resulting in a twodimensional space containing the affective terms. This space was then scaled to fit the dimensions pleasure-displeasure (i.e., valence) and degree of arousal. Some researchers also use a third dimension-namely, dominance (Russell & Mehrabian, 1977). However, arousal and valence appear to be sufficient to explain most of the variance of affective scales (Lang, 1995). Moreover, the use of a computer monitor restricts the software to two dimensions, of which valence and arousal appear to be the most important and universal (Russell, 1983). The use of other dimensions, such as pleasantness and liking, has recently been discussed by Ritossa and Rickard (2004).With respect to emotions in music, Schubert used Russell's model of two basic emotional dimensions in his Emotionspace Lab (2DES;Schubert, 1996). He proved, as Russell did, the validity and reliability of arousal and valence ...

Oudjat: A configurable and usable annotation tool for the study of facial expressions of emotion

“…Gtrace software (an upgrade of FeelTrace Cowie et al, 2011Cowie et al, , 2000 Freeware https://sites.google.com/site/roddycowie/ work-resources Dimensions CARMA (Girard, 2014) Freeware https://carma.codeplex.com/ Dimensions EMuJoy (Nagel et al, 2007) Freeware Not available Dimensions CVML Video Affective Annotation interface (Soleymani and Larson, 2011;Soleymani et al, 2012) Freeware Not available Dimensions…”

Music emotion ranking

“…For simplicity in execution and analysis, most existing psychological studies [6]- [8] adopted rating measure to help subjects track the emotion variation of a music piece as it unfolds in time. In our previous work [4] of building an MER system that views emotion from the continuous perspective and predicts the representative emotion of a song, we also adopted the rating measure and asked subjects to rate arousal and valence (separately) from -1.0 to 1.0 in 11 ordinal levels.…”

“…The first issue is related to the annotation of emotion. The rating measure (either the standard ordinal rating scale or the graphic rating scale) commonly adopted in the literature [3]- [8] may not be an appropriate choice due to the heavy cognitive load for human to directly express a continuum of emotions. In addition, it cannot ensure that the scale is consistent between and within subjects [9].…”

Can computer mice be used as low-cost devices for the acquisition of planar human movement velocity signals?

“…In this paper, we study the possibility of using this device as a cheap but precise and reliable acquisition tool for studying the velocity of human arm motion. Many disciplines could benefit from using a computer mouse for such a purpose, including biomedical engineering (O'Reilly, Plamondon, Lebrun, Clément, & Mathieu, 2009), experimental psychology (Brenner & Smeets, 2003;Jansen, Blackwell, & Marriott, 2003;Morein-Zamir, Chua, Franks, Nagelkerke, & Kingstone, 2006;Nagel, Kopiez, Grewe, & Altenmuller, 2007;Thompson, McConnell, Slocum, & Bohan, 2007;Walker, Meyer, & Smelcer, 1993), the neurosciences (Ogawa, Inui, & Sugio, 2007;Roby-Brami & Burnod, 1995;Young, Pratt, & Chau, 2008), rehabilitation engineering ), ergonomics (Phillips & Triggs, 2001), and so on.…”

“…Some human movement experiments (Brenner & Smeets, 2003;Faustina, Simeon, Patrick, & John, 2004;Jansen et al, 2003;Kisacanin, Agarwal, Taber, & Hier, 2000;Morein-Zamir et al, 2006;Nagel et al, 2007;Nijhof, 2003;Ogawa et al, 2007;Phillips & Triggs, 2001;Pratt & Turk-Browne, 2003;Roby-Brami & Burnod, 1995;Slijper, Richter, Over, Smeets, & Frens, 2009;Thompson et al, 2007;Walker et al, 1993) have used computer mice as well, but without detailed analysis of signal acquisition reliability. In these studies, mouse signals seem to have been considered acceptable a priori.…”