Measuring Cognitive Load Using In-Game Metrics of a Serious Simulation Game

Abstract

Serious games have become an important tool to train individuals in a range of different skills. Importantly, serious games or gamified scenarios allow for simulating realistic time-critical situations to train and also assess individual performance. In this context, determining the user’s cognitive load during (game-based) training seems crucial for predicting performance and potential adaptation of the training environment to improve training effectiveness. Therefore, it is important to identify in-game metrics sensitive to users’ cognitive load. According to Barrouillets’ time-based resource-sharing model, particularly relevant for measuring cognitive load in time-critical situations, cognitive load does not depend solely on the complexity of actions but also on temporal aspects of a given task. In this study, we applied this idea to the context of a serious game by proposing in-game metrics for workload prediction that reflect a relation between the time during which participants’ attention is captured and the total time available for the task at hand. We used an emergency simulation serious game requiring management of time-critical situations. Forty-seven participants completed the emergency simulation and rated their workload using the NASA-TLX questionnaire. Results indicated that the proposed in-game metrics yielded significant associations both with subjective workload measures as well as with gaming performance. Moreover, we observed that a prediction model based solely on data from the first minutes of the gameplay predicted overall gaming performance with a classification accuracy significantly above chance level and not significantly different from a model based on subjective workload ratings. These results imply that in-game metrics may qualify for a real-time adaptation of a game-based learning environment.

Effects of Cognitive Load Level on Students’ Attitude towards the Gamified Course

“…The contrasting results of the mentioned studies point out that it is important to decrease cognitive load level whereas increase learning performance of students in gamified activities (Shaban et al, 2021). Regardless of the reason of cognitive load, it should be applied the learning activities by considering the critical importance to manage it (Ayres, 2020;Sevcenko et al, 2021).…”

“…However, Turan et al (2016) determined that students had highly cognitive load level in gamification process. Sevcenko et al (2021) reported that cognitive load depended on not only the complexity of the activity but also timing of the task as it might quickly change when performing the task. They also stressed that even if each student was performing the same task, they might make different mental effort.…”

“…However, many of these studies compare the students' attitude towards gamified and non-gamified course. Additionally, most of the studies about gamification focus on students' affective aspect, while just a few studies underline that it needs to be considered the cognitive aspect in gamification as well (Sevcenko et al, 2021;Shaban et al 2021;Su, 2016;Wu, 2018).…”

Design and Development of a Scaffolding-Based Mindtool for Gamified Learning Classrooms

“…The game information module contains game information such as game goals, game rules (steps), and points. This module is designed to reduce the cognitive load on students' memory for understanding complex game rules (Sevcenko et al, 2021). The game information module also includes a real-time display of students' current points, so that students can self-review their learning performance and reflect on their points (Kim & Castelli, 2021).…”

“…Each game element and task content have an impact on students' cognitive load. Overly complicated rules or learning content can cause too much cognitive load for students to enjoy the activity or game and can thus prevent them from learning (Sevcenko et al, 2021). In addition, students' prior knowledge is another a factor that affects the cognitive load during game play.…”