Heart rate variability analysis for the assessment of immersive emotional arousal using virtual reality: Comparing real and virtual scenarios

Abstract

Many affective computing studies have developed automatic emotion recognition models, mostly using emotional images, audio and videos. In recent years, virtual reality (VR) has been also used as a method to elicit emotions in laboratory environments. However, there is still a need to analyse the validity of VR in order to extrapolate the results it produces and to assess the similarities and differences in physiological responses provoked by real and virtual environments. We investigated the cardiovascular oscillations of 60 participants during a free exploration of a real museum and its virtualisation viewed through a head-mounted display. The differences between the heart rate variability features in the high and low arousal stimuli conditions were analysed through statistical hypothesis testing; and automatic arousal recognition models were developed across the real and the virtual conditions using a support vector machine algorithm with recursive feature selection. The subjects’ self-assessments suggested that both museums elicited low and high arousal levels. In addition, the real museum showed differences in terms of cardiovascular responses, differences in vagal activity, while arousal recognition reached 72.92% accuracy. However, we did not find the same arousal-based autonomic nervous system change pattern during the virtual museum exploration. The results showed that, while the direct virtualisation of a real environment might be self-reported as evoking psychological arousal, it does not necessarily evoke the same cardiovascular changes as a real arousing elicitation. These contribute to the understanding of the use of VR in emotion recognition research; future research is needed to study arousal and emotion elicitation in immersive VR.

Multimodal Assessment of Changes in Physiological Indicators when Presenting a Video Fragment on Screen (2D) versus a VR (3D) Environment

“…Accordingly, we observed a decrease in the LF/HF ratio (with the LF% value unchanged), indicating the predominance of parasympathetic influences on heart rhythm regulation under the indicated conditions. Similar results have been demonstrated by many [ 34 , 42 ]. In particular, in the study of Sokhadze [ 42 ], where the participants were presented with images of mutilated bodies, there was an increase in HF power and a decrease in the LF/HF ratio.…”

“…They have shown that positive emotions evoke significantly more alpha activity than negative and neutral emotions when shown videos of different emotional content under VR conditions. When studying various physiological measures in VR, Marín-Morales et al [ 26 , 34 , 35 ] found that the most pronounced changes in characteristics were related to EEG rather than HR.…”

Heart Rate Variability in Healthy Subjects During Monitored, Short-Term Stress Followed by 24-hour Cardiac Monitoring

“…Our results showed that DFA α 1 as a non-linear index of HRV did not distinguish the different affective image regimes under ultra-short recordings. Although previous work showed that there can be differences in HRV measures when comparing neutral and arousing sessions ( Valenza et al, 2012 ) or watching positive and negative videos ( Barquero-Pérez et al, 2020 ; Ghiasi et al, 2020 ), our findings are consistent with other works that showed the linear HRV metric RMSSD did not distinguish between positive and negative emotions using IAPS ( Schippers et al, 2018 ), and non-linear HRV metric DFA α 1 did not distinguish different emotionally arousing settings ( Marín-Morales et al, 2021 ). We hypothesize that because pure regime participants knew to expect images of the same type, there may not have been a sufficient change in emotional context to create significant variation in their heart rate.…”

Virtual and Reality: A Neurophysiological Pilot Study of the Sarcophagus of the Spouses

“…One of those studies, aimed at investigating the elicitation of the same emotional response based on both electroencephalographic (EEG) and electrocardiographic biosignals recorded during the free exploration of an art museum and its 3D immersive reconstruction, found 95.27% accuracy for the real vs. virtual classifier using only EEG mean phase coherency features [ 13 ]. Furthermore, it was evidenced self-reported psychological arousal for both real and VR museums, but only in the real one were reported differences in terms of cardiovascular responses, suggesting that the VR reconstruction of a real environment might be self-reported as psychologically arousing but might not necessarily evoke the same cardiovascular changes as a real arousing elicitation [ 14 ].…”