Real-Time Emotional State Estimator for Adaptive Virtual Reality Stimulation

Abstract

The paper presents design and evaluation of emotional state estimator based on artificial neural networks for physiology-driven adaptive virtual reality (VR) stimulation. Real-time emotional state estimation from physiological signals enables adapting the stimulations to the emotional response of each individual. Estimation is first evaluated on artificial subjects, which are convenient during software development and testing of physiology-driven adaptive VR stimulation. Artificial subjects are implemented in the form of parameterized skin conductance and heart rate generators that respond to emotional inputs. Emotional inputs are a temporal sequence of valence/arousal annotations, which quantitatively express emotion along unpleasant-pleasant and calm-aroused axes. Preliminary evaluation of emotional state estimation is also performed with a limited set of humans. Human physiological signals are acquired during simultaneous presentation of static pictures and sounds from valence/arousalannotated International Affective Picture System and International Affective Digitized Sounds databases.

Remote assessment of the Heart Rate Variability to detect mental stress

“…Recognizing an emotion by its physiological signature is a field of research that presents a particular interest on the last ten years. Understanding emotions can be useful, particularly during virtual reality (VR) therapies, where emotions are feedbacks that regulate the virtual environment level and intensity [1]. These physiology-driven virtual scenes are particularly employed to treat patients with anxiety disorders and phobia, like soldiers and war veterans [2].…”

Development of Acoustic Model for Croatian Language Using HTK

“…keywords and n-grams) [12,13]. The emotional state estimator is part of an ongoing research project: Adaptive control of virtual reality scenarios in therapy of posttraumatic stress disorder (PTSD), at University of Zagreb [14,15].…”

Physiology-Driven Adaptive Virtual Reality Stimulation for Prevention and Treatment of Stress Related Disorders

“…Our initial experiments related to real-time emotional state estimation from skin conductance and heart rate during emotion elicitation with IAPS pictures and IADS sounds have applied the artificial neural networks. 28 The relatively low accuracy of results obtained by participant-independent emotional state estimation has emphasized the importance of customizing the estimator to every individual. Estimation accuracy may also benefit from increasing the number of signals and computed features as well as from fusions of outputs of multiple specialized estimators running simultaneously.…”

“…As such, the estimator may also be useful in a standalone fashion, not only in SIT and PTSD exposure therapy but in a broader range of human-computer interaction applications following Picard's 19 affective computing paradigm. As briefly reviewed in Kukolja et al, 28 the emotional state estimator can be based on a variety of methods. Our initial experiments related to real-time emotional state estimation from skin conductance and heart rate during emotion elicitation with IAPS pictures and IADS sounds have applied the artificial neural networks.…”