Predicting musically induced emotions from physiological inputs: linear and neural network models

Abstract

Listening to music often leads to physiological responses. Do these physiological responses contain sufficient information to infer emotion induced in the listener? The current study explores this question by attempting to predict judgments of “felt” emotion from physiological responses alone using linear and neural network models. We measured five channels of peripheral physiology from 20 participants—heart rate (HR), respiration, galvanic skin response, and activity in corrugator supercilii and zygomaticus major facial muscles. Using valence and arousal (VA) dimensions, participants rated their felt emotion after listening to each of 12 classical music excerpts. After extracting features from the five channels, we examined their correlation with VA ratings, and then performed multiple linear regression to see if a linear relationship between the physiological responses could account for the ratings. Although linear models predicted a significant amount of variance in arousal ratings, they were unable to do so with valence ratings. We then used a neural network to provide a non-linear account of the ratings. The network was trained on the mean ratings of eight of the 12 excerpts and tested on the remainder. Performance of the neural network confirms that physiological responses alone can be used to predict musically induced emotion. The non-linear model derived from the neural network was more accurate than linear models derived from multiple linear regression, particularly along the valence dimension. A secondary analysis allowed us to quantify the relative contributions of inputs to the non-linear model. The study represents a novel approach to understanding the complex relationship between physiological responses and musically induced emotion.

Higher-order Multivariable Polynomial Regression to Estimate Human Affective States

“…5 ). This was in line with prior researches 5 6 13 14 31 34 . In the present study, a HMPM can give a better description of nonlinearities of the SCR affective process than what a linear model can do.…”

“…Further more, continuously estimating individual affect can supply more quantitative results than what classification procedures can supply. Methods of estimating human affective states, such as multivariate linear-regression analysis (MLR), partial least-square estimation (PLS), genetic algorithm optimized support vector regression (GA-SVR), artificial neural network (ANN), bidirectional long short-term memory neural networks (LSTM-NNs), fuzzy logical analysis (FLA), and sequence Bayessian analysis (SBA), have been proposed in the past decade 28 29 30 31 32 33 34 . Although their performances differ with each other, these computational models achieve relative good results to some extent, and supply us a good prospect in affective estimation.…”

Perception and Modeling of Affective Qualities of Musical Instrument Sounds across Pitch Registers

“…Previous research has shown that non-linear methods can be particularly useful in situations where linear methods are insufficient to model the relationship between dependent and independent variables. Artificial neural networks may be used as a non-linear regression method (Coutinho and Cangelosi, 2011 ; Russo et al, 2013 ; Vempala and Russo, 2013 ) to predict valence, tension arousal, and energy arousal ratings using timbre descriptors.…”

Autonomic Effects of Music in Health and Crohn's Disease: The Impact of Isochronicity, Emotional Valence, and Tempo

“…Generally, emotional arousal is associated with sympathetic ANS activity [ 1 ], thus leading to an increase in HR, whereas parasympathetic ANS activity leads to a decrease in HR [ 24 ]. Accordingly, several studies reported (trends towards) higher HR with arousing music compared to tranquilizing or less arousing music [ 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 ]. However, when comparing HR during music listening to baseline measurements, previous studies do not provide a consistent picture: Although some studies demonstrated an increase in HR with arousing music and a decrease in HR with tranquilizing music [ 25 , 27 ], others reported increases in HR with arousing as well as tranquilizing music [ 26 , 34 ], decreases in HR with tranquilizing as well as arousing music [ 29 , 35 ], or differences in HR despite similar music-evoked arousal [ 36 ].…”