Virtual reality facial emotion recognition in social environments: An eye-tracking study

Geraets, Chris; Tuente, Stéphanie Klein; Lestestuiver, Bart; Beilen, Marije van; Nijman, Saskia Anne; Marsman, Jan-Bernard; Veling, Wim

doi:10.1016/j.invent.2021.100432

Cited by 44 publications

(31 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We found a main effect of age on emotion recognition accuracy, replicating other studies. 18 , 19 Contrary to previous meta-analyses on (non-VR) SCT finding associations between emotion perception and gender, hospitalization status, clinical symptoms and antipsychotic treatment, 20 we found no other predictors or moderators of accuracy. This could be due to the absence of an effect, interference due to the interrelatedness of time and task difficulty, and/or insufficient power to detect moderation effects.…”

Section: Discussioncontrasting

confidence: 99%

“…The question, therefore, remains to what degree improvements in VR generalize to performance on other social cognitive tasks and daily life. As pointed out in the Introduction, a study 18 using the same VR environment found that performance and confusion patterns across the VR task, a photo task (Ekman 60 Faces Test), and a video task 34 were comparable in healthy individuals.…”

Section: Discussionmentioning

confidence: 87%

“…Given the lack of effects of DiSCoVR on social cognition measures, we investigated whether a treatment effect was present in VR, and whether this potential effect generalized. Given the dearth of knowledge on factors contributing to treatment efficacy, we opted to use an exploratory approach, selecting variables from our data that have previously been investigated as predictors of emotion recognition 18–20 and moderators of SCT efficacy. 6 , 8 , 12 We investigated the following research questions:…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Trajectories of Emotion Recognition Training in Virtual Reality and Predictors of Improvement for People with a Psychotic Disorder

Nijman

Veling

Timmerman

et al. 2023

Cyberpsychology, Behavior, and Social Networking

View full text Add to dashboard Cite

Meta-analyses have found that social cognition training (SCT) has large effects on the emotion recognition ability of people with a psychotic disorder. Virtual reality (VR) could be a promising tool for delivering SCT. Presently, it is unknown how improvements in emotion recognition develop during (VR-)SCT, which factors impact improvement, and how improvements in VR relate to improvement outside VR. Data were extracted from task logs from a pilot study and randomized controlled trials on VR-SCT ( n = 55). Using mixed-effects generalized linear models, we examined the: (a) effect of treatment session (1–5) on VR accuracy and VR response time for correct answers; (b) main effects and moderation of participant and treatment characteristics on VR accuracy; and (c) the association between baseline performance on the Ekman 60 Faces task and accuracy in VR, and the interaction of Ekman 60 Faces change scores (i.e., post-treatment − baseline) with treatment session. Accounting for the task difficulty level and the type of presented emotion, participants became more accurate at the VR task ( b = 0.20, p < 0.001) and faster ( b = −0.10, p < 0.001) at providing correct answers as treatment sessions progressed. Overall emotion recognition accuracy in VR decreased with age ( b = −0.34, p = 0.009); however, no significant interactions between any of the moderator variables and treatment session were found. An association between baseline Ekman 60 Faces and VR accuracy was found ( b = 0.04, p = 0.006), but no significant interaction between difference scores and treatment session. Emotion recognition accuracy improved during VR-SCT, but improvements in VR may not generalize to non-VR tasks and daily life.

show abstract

Section: Discussioncontrasting

confidence: 99%

Section: Discussionmentioning

confidence: 87%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Trajectories of Emotion Recognition Training in Virtual Reality and Predictors of Improvement for People with a Psychotic Disorder

Nijman

Veling

Timmerman

et al. 2023

Cyberpsychology, Behavior, and Social Networking

View full text Add to dashboard Cite

show abstract

“…Traditional therapeutic tools are limited to psychotherapy and drug therapy. The development of VR technology has promoted its application in this field, including a series of studies aimed at assessing and improving symptoms of schizophrenia [ 63 ], as well as VR emotion recognition tasks aimed at treating facial emotion recognition disorders in patients with neurological and psychiatric disorders in recent years [ 64 , 65 ]. To develop VR into a mature tool for improving mental health, researchers need to integrate knowledge from health care, computer science, neuroscience, psychology, social cognition, multisensory perception, and multimedia development in an interdisciplinary approach [ 27 ] and specifically combine user needs.…”

Section: Discussionmentioning

confidence: 99%

Virtual Reality Technology in Cognitive Rehabilitation Application: Bibliometric Analysis

He¹,

Cao²,

Le³

et al. 2022

JMIR Serious Games

View full text Add to dashboard Cite

Background In recent years, with the development of computer science and medical science, virtual reality (VR) technology has become a promising tool for improving cognitive function. Research on VR-based cognitive training has garnered increasing attention. Objective This study aimed to investigate the application status, research hot spots, and emerging trends of VR in cognitive rehabilitation over the past 20 years. Methods Articles on VR-based cognitive rehabilitation from 2001 to 2021 were retrieved from the Web of Science Core Collection. CiteSpace software was used for the visual analysis of authors and countries or regions, and Scimago Graphica software was used for the geographic visualization of published countries or regions. Keywords were clustered using the gCLUTO software. Results A total of 1259 papers were included. In recent years, research on the application of VR in cognitive rehabilitation has been widely conducted, and the annual publication of relevant literature has shown a positive trend. The main research areas include neuroscience and neurology, psychology, computer science, and rehabilitation. The United States ranked first with 328 papers, and Italy ranked second with 140 papers. Giuseppe Riva, an Italian academic, was the most prolific author with 29 publications. The most frequently cited reference was “Using Reality to Characterize Episodic Memory Profiles in Amnestic Mild Cognitive Impairment and Alzheimer’s Disease: Influence of Active and Passive Encoding.” The most common keywords used by researchers include “virtual reality,” “cognition,” “rehabilitation,” “performance,” and “older adult.” The largest source of research funding is from the public sector in the United States. Conclusions The bibliometric analysis provided an overview of the application of VR in cognitive rehabilitation. VR-based cognitive rehabilitation can be integrated into multiple disciplines. We conclude that, in the context of the COVID-19 pandemic, the development of VR-based telerehabilitation is crucial, and there are still many problems that need to be addressed, such as the lack of consensus on treatment methods and the existence of safety hazards.

show abstract

“…In the field of perspective taking research, task stimuli can vary between pictorial representations of people [ 68 ] to live tasks presented by a confederate [ 69 ]. Similarly, in emotion recognition research, the task stimuli can vary from static, regimented presentations of actors such as those outlined by Eckman [ 70 ] to more complex emotional stimuli presented via video presentations or, relatively recently, via virtual reality avatars [ 37 , 71 ]. Yet, despite the clear differences in social complexity in the stimuli used in these paradigms, it is not yet understood if these differences lead to empirical alterations of behaviour.…”

Section: Moving Beyond Social Attention: the Effect Of Social Presenc...mentioning

confidence: 99%

I see you, you see me: the impact of social presence on social interaction processes in autistic and non-autistic people

Freeth

Morgan

2023

Phil. Trans. R. Soc. B

View full text Add to dashboard Cite

Environments that require social interaction are complex, challenging and sometimes experienced as overwhelming by autistic people. However, all too often theories relating to social interaction processes are created, and interventions are proposed, on the basis of data collected from studies that do not involve genuine social encounters nor do they consider the perception of social presence to be a potentially influential factor. In this review, we begin by considering why face-to-face interaction research is important in this field. We then discuss how the perception of social agency and social presence can influence conclusions about social interaction processes. We then outline some insights gained from face-to-face interaction research conducted with both autistic and non-autistic people. We finish by considering the impact of social presence on cognitive processes more broadly, including theory of mind. Overall, we demonstrate that choice of stimuli in studies assessing social interaction processes has the potential to substantially alter conclusions drawn. Ecological validity matters and social presence, in particular, is a critical factor that fundamentally impacts social interaction processes in both autistic and non-autistic people. This article is part of a discussion meeting issue ‘Face2face: advancing the science of social interaction’.

show abstract

Virtual reality facial emotion recognition in social environments: An eye-tracking study

Cited by 44 publications

References 44 publications

Trajectories of Emotion Recognition Training in Virtual Reality and Predictors of Improvement for People with a Psychotic Disorder

Trajectories of Emotion Recognition Training in Virtual Reality and Predictors of Improvement for People with a Psychotic Disorder

Virtual Reality Technology in Cognitive Rehabilitation Application: Bibliometric Analysis

I see you, you see me: the impact of social presence on social interaction processes in autistic and non-autistic people

Contact Info

Product

Resources

About