Total Immersion: Designing for Affective Symbiosis in a Virtual Reality Game with Haptics, Biosensors, and Emotive Agents

Abstract

Abstract. Affective symbiosis for human-computer interaction refers to the dynamic relationship between the user and affective virtual agents. In order to facilitate a true, immersive experience, we believe that it is necessary to adapt the presentation of an affective agent to the user's affective state. Investigating the experience, behavior, and physiological correlates of affective events, such as winning and losing during a competitive game, therefore might be used to adapt the agent's emotional states and system events. An experimental virtual reality game environment was designed as a stepping stone toward a system to demonstrate affective symbiosis. Users were invited to play a game of air hockey with affective agents in virtual reality. We collected the electrocardiography, electrodermal activity, and postural data, as well as self-reports, to investigate how emotional events affected physiology, behavior, and experience. The users were found to be engaged in the competition strongly while only paying limited attention to their adversaries' emotional expressions. We discuss how game events are much stronger causes for affective responses, with the physiological effects of winning and losing becoming more enhanced as the game progresses. We discuss how an affective, symbiotic system could implement both game events and dynamic, affective agents to create a system for total immersion.

Heart Rate Detection from the Supratrochlear Vessels using a Virtual Reality Headset integrated PPG Sensor

“…Figures 4 and 5 show mean heart rates for each time window. A repeated measures ANOVA with the factors recording methods (ECG vs PPG), condition (sitting vs. standing) and time window (1)(2)(3)(4)(5)(6)(7)(8) revealed no significant difference between the ECG and PPG methods in heart rate measures, F(1,15) = .010, p=.923. Sitting vs standing condition were significantly different F(1,15) = 19.79, p<.001.…”

“…Virtual reality (VR) has found its way into mainstream research and its use is steadily increasing due to numerous benefits, such as precise control over stimuli, ability to display objects in 3 dimensions [17] and increased ecological validity [12] to name a few. Many studies have quantified the affective responses to VR environments using biosensors [1,9,15,18]. ECG measurements, via individual sensors on the chest or integrated in a belt set-up, have been a standard practice for heart rate measurements [2].…”

Erasmus XR - Immersive experiences in European academic institutions

“…Erasmus XR will also elaborate open educational resources (OER) [5] in the field of experience management [6], [7] and culture with the use of immersive technologies. Erasmus XR's aim is to help building an open attitude [8] of students and academic staff towards the application of new technologies in the humanities. Due to the technological aspect [9] of the project, there will be a bridging of the gap in terms of access to solutions and their use by groups which have not had the opportunity to use them so far (in particular in the field of culture).…”

Emotion Recognition in Virtual Reality: Investigating the Effect of Gameplay Variations on Affective Responses

“…Physiological signals like heart rate, skin conductance, and EEG are analyzed in VR to interpret users' emotional responses, enabling VR environments to adapt to the user's emotional state (Bayro, Ghasemi, & Jeong, 2022;Darzi & Novak, 2017). VR games, such as Pong, serve as controlled environments to study human emotions and behaviors, with the data used to create systems that can identify and respond to emotional cues (Ahmed, Harjunen, Jacucci, Ravaja, & Spapé, 2017). Changing game characteristics can shed light on how they impact a player's emotions, thereby informing the development of affective computing systems.…”