Evoking Physiological Synchrony and Empathy Using Social VR With Biofeedback

Salminen, Matti; Järvelä, Sanna; Ruonala, Antti; Harjunen, Ville; Jacucci, Giulio; Hamari, Juho; Ravaja, Niklas

doi:10.1109/taffc.2019.2958657

Cited by 46 publications

(33 citation statements)

References 62 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Dyads could then take steps to try to increase engagement by, e.g., changing conversation topics or shifting the balance of conversation from one person to the other. Such real-time feedback has been proposed by multiple researchers, and very simple versions of it have been implemented – for example, displaying the other participants’ heart rates, respiration rates or brain waves and allowing the viewer to make their own interpretations ( Frey, 2016 ; Liu et al, 2017 ; Salminen et al, 2019 ). By fusing information from multiple physiological signals into an overall engagement estimate, our approach may allow both more accurate and more easily interpretable feedback.…”

Section: Discussionmentioning

confidence: 99%

Automatic Estimation of Interpersonal Engagement During Naturalistic Conversation Using Dyadic Physiological Measurements

et al. 2021

View full text Add to dashboard Cite

Physiological responses of two interacting individuals contain a wealth of information about the dyad: for example, the degree of engagement or trust. However, nearly all studies on dyadic physiological responses have targeted group-level analysis: e.g., correlating physiology and engagement in a large sample. Conversely, this paper presents a study where physiological measurements are combined with machine learning algorithms to dynamically estimate the engagement of individual dyads. Sixteen dyads completed 15-min naturalistic conversations and self-reported their engagement on a visual analog scale every 60 s. Four physiological signals (electrocardiography, skin conductance, respiration, skin temperature) were recorded, and both individual physiological features (e.g., each participant’s heart rate) and synchrony features (indicating degree of physiological similarity between two participants) were extracted. Multiple regression algorithms were used to estimate self-reported engagement based on physiological features using either leave-interval-out crossvalidation (training on 14 60-s intervals from a dyad and testing on the 15th interval from the same dyad) or leave-dyad-out crossvalidation (training on 15 dyads and testing on the 16th). In leave-interval-out crossvalidation, the regression algorithms achieved accuracy similar to a ‘baseline’ estimator that simply took the median engagement of the other 14 intervals. In leave-dyad-out crossvalidation, machine learning achieved a slightly higher accuracy than the baseline estimator and higher accuracy than an independent human observer. Secondary analyses showed that removing synchrony features and personality characteristics from the input dataset negatively impacted estimation accuracy and that engagement estimation error was correlated with personality traits. Results demonstrate the feasibility of dynamically estimating interpersonal engagement during naturalistic conversation using physiological measurements, which has potential applications in both conversation monitoring and conversation enhancement. However, as many of our estimation errors are difficult to contextualize, further work is needed to determine acceptable estimation accuracies.

show abstract

Section: Discussionmentioning

confidence: 99%

Automatic Estimation of Interpersonal Engagement During Naturalistic Conversation Using Dyadic Physiological Measurements

et al. 2021

View full text Add to dashboard Cite

show abstract

“…From psychological studies on empathy, we know that it is important to differentiate between the two sides of empathy: spontaneous empathic interest in the other, which is close to an altruistic orientation, and the negative side, defined in the literature as personal distress in reaction to another person's need or suffering. This double nature of empathy has been also explored in psychophysiological studies, in which scholars pointed out that empathy can be associated with both left and right EEG asymmetry as it is an emotional response to the other's need (left frontal asymmetry) and, at the same time, involves emotional sharing of the other's pain (right frontal asymmetry) [20].…”

Section: Intergroup Helping and Empathymentioning

confidence: 99%

Prosocial Virtual Reality, Empathy, and EEG Measures: A Pilot Study Aimed at Monitoring Emotional Processes in Intergroup Helping Behaviors

et al. 2020

View full text Add to dashboard Cite

During a non-invasive procedure, participants both helped and helped by a confederate with features that create social distance (membership in an ethnic outgroup or another social group). For this purpose, we created a set of virtual scenarios in which the confederate's ethnicity (white vs. black) and appearance (business man vs. beggar, with casual dress as a control condition) were crossed. The study aimed to explore how the emotional reactions of participants changed according to the confederate's status signals as well as signals that they belong to the same or a different ethnic group. Participants' alertness, calmness, and engagement were monitored using electroencephalogram (EEG) during the original virtual reality (VR) video sessions. Participants' distress and empathy when exposed to helping interactions were self-assessed after the VR video sessions. The results pointed out that, irrespective of whether they helped the confederate or were helped by him/her, white participants showed higher levels of alertness when exposed to helping interactions involving a white beggar or a black businessman, and their emotional calmness and engagement were higher when interacting with a black beggar or a white businessman. The results for self-assessed distress and empathy followed the same tendency, indicating how physiological and self-assessed measures can both contribute to a better understanding of the emotional processes in virtual intergroup helping situations. Based on the presented results, the methodological and practical implications of VR in terms of enhancing self-reflective capacities in intergroup helping processes are discussed.

show abstract

“…By merging the experience of the avatar of another individuals and the information about his/her internal states it is possible to step into the shoes of the other individual [62]. A recent study, used interpersonally shared biofeedback (visualised respiration rate and frontal asymmetry of electroencephalography, EEG) to enhance synchrony between the users' physiological activity and perceived empathy towards the other during a compassion meditation exercise carried out in a social VR setting [63].…”

Section: Clinical Xr: Augmenting Multisensory Bodily Contentsmentioning

confidence: 99%

Extended Reality for the Clinical, Affective, and Social Neurosciences

2020

View full text Add to dashboard Cite

Brain science research often involves the use of low-dimensional tools and stimuli that lack several of the potentially valuable features of everyday activities and interactions. Although this research has provided important information about cognitive, affective, and social processes for both clinical and nonclinical populations, there is growing interest in high-dimensional simulations that extend reality. These high-dimensional simulations involve dynamic stimuli presented serially or concurrently to permit the assessment and training of perceivers’ integrative processes over time. Moreover, high-dimensional simulation platforms can contextually restrain interpretations of cues about a target’s internal states. Extended reality environments extend assessment and training platforms that balance experimental control with emotionally engaging background narratives aimed at extending the affective experience and social interactions. Herein, we highlight the promise of extended reality platforms for greater ecological validity in the clinical, affective, and social neurosciences.

show abstract

Evoking Physiological Synchrony and Empathy Using Social VR With Biofeedback

Cited by 46 publications

References 62 publications

Automatic Estimation of Interpersonal Engagement During Naturalistic Conversation Using Dyadic Physiological Measurements

Automatic Estimation of Interpersonal Engagement During Naturalistic Conversation Using Dyadic Physiological Measurements

Prosocial Virtual Reality, Empathy, and EEG Measures: A Pilot Study Aimed at Monitoring Emotional Processes in Intergroup Helping Behaviors

Extended Reality for the Clinical, Affective, and Social Neurosciences

Contact Info

Product

Resources

About