Enhancing User Engagement in Immersive Games through Multisensory Cues

Kruijff, Ernst; Marquardt, Alexander; Trepkowski, Christina; Schild, Jonas; Hinkenjann, André

doi:10.1109/vs-games.2015.7295773

Cited by 12 publications

(6 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The study showed that the fusion of individual input signals can increase the accuracy of the engagement detection. Similarly, other researchers have proven that the accuracy in affect recognition tasks is increased by using multi-sensor approaches [48] [49].…”

Section: Related Workmentioning

confidence: 87%

Multimodal Student Engagement Recognition in Prosocial Games

et al. 2018

View full text Add to dashboard Cite

In this paper we address the problem of recognizing student engagement in prosocial games by exploiting engagement cues from different input modalities. Since engagement is a multifaceted phenomenon with different dimensions, i.e., behavioral, cognitive and affective, we propose the modeling of student engagement using real-time data from both the students and the game. More specifically, we apply body motion and facial expression analysis to identify the affective state of students, while we extract features related to their cognitive and behavioral engagement based on the analysis of their interaction with the game. For the automatic recognition of engagement, we adopt a machine learning approach based on artificial neural networks, while for the annotation of the engagement data, we introduce a novel approach based on the use of games with different degrees of challenge in conjunction with a retrospective self-reporting method. To evaluate the proposed methodology, we conducted real-life experiments in four classes, in three primary schools, with 72 students and 144 gameplay recordings in total. Experimental results show the great potential of the proposed methodology, which improves the classification accuracy of the three distinct dimensions with a detection rate of 85%. A detailed analysis of the role of each component of the Game Engagement Questionnaire (GEQ), i.e., immersion, presence, flow and absorption, in the classification process is also presented in this paper.

show abstract

Section: Related Workmentioning

confidence: 87%

Multimodal Student Engagement Recognition in Prosocial Games

et al. 2018

View full text Add to dashboard Cite

show abstract

“…Multisensory feedback can enhance many of the high and complex cognitive processes involved in VR [41]. Particularly, multimodality can trigger different emotional responses in immersive games: Kruijff et al [94] investigated those emotional effects and proposed guidelines that can be applied to reproduce diverse emotional responses in multimodal games. Within the wide area of entertainment, cinematographic and narrative experiences in VR have been emerging during the last years.…”

Section: Entertainmentmentioning

confidence: 99%

Multimodality in VR: A Survey

et al. 2022

View full text Add to dashboard Cite

Virtual reality (VR) is rapidly growing, with the potential to change the way we create and consume content. In VR, users integrate multimodal sensory information they receive, to create a unified perception of the virtual world. In this survey, we review the body of work addressing multimodality in VR, and its role and benefits in user experience, together with different applications that leverage multimodality in many disciplines. These works thus encompass several fields of research, and demonstrate that multimodality plays a fundamental role in VR; enhancing the experience, improving overall performance, and yielding unprecedented abilities in skill and knowledge transfer.

show abstract

“…As some of those aforementioned works have shown, multisensory cues can trigger different users' emotional responses in immersive games. Kruijff et al [83] reported another work where they investigated those effects and proposed guidelines that can be applied to reproduce similar emotional responses. Whether because of emotional responses or because of the complexity of the games, cognitive loads can be quite high.…”

Section: Entertainmentmentioning

confidence: 99%

Multimodality in VR: A survey

Martin,

Malpica,

Gutierrez

et al. 2021

Preprint

View full text Add to dashboard Cite

Virtual reality has the potential to change the way we create and consume content in our everyday life. Entertainment, training, design and manufacturing, communication, or advertising are all applications that already benefit from this new medium reaching consumer level. VR is inherently different from traditional media: it offers a more immersive experience, and has the ability to elicit a sense of presence through the place and plausibility illusions. It also gives the user unprecedented capabilities to explore their environment, in contrast with traditional media. In VR, like in the real world, users integrate the multimodal sensory information they receive to create a unified perception of the virtual world. Therefore, the sensory cues that are available in a virtual environment can be leveraged to enhance the final experience. This may include increasing realism, or the sense of presence; predicting or guiding the attention of the user through the experience; or increasing their performance if the experience involves the completion of certain tasks. In this state-of-the-art report, we survey the body of work addressing multimodality in virtual reality, its role and benefits in the final user experience. The works here reviewed thus encompass several fields of research, including computer graphics, human computer interaction, or psychology and perception. Additionally, we give an overview of different applications that leverage multimodal input in areas such as medicine, training and education, or entertainment; we include works in which the integration of multiple sensory information yields significant improvements, demonstrating how multimodality can play a fundamental role in the way VR systems are designed, and VR experiences created and consumed.

show abstract

Enhancing User Engagement in Immersive Games through Multisensory Cues

Cited by 12 publications

References 52 publications

Multimodal Student Engagement Recognition in Prosocial Games

Multimodal Student Engagement Recognition in Prosocial Games

Multimodality in VR: A Survey

Multimodality in VR: A survey

Contact Info

Product

Resources

About