Capturing Sequences of Learners' Self-Regulatory Interactions With Instructional Material During Game-Based Learning Using Auto-Recurrence Quantification Analysis

Abstract

Undergraduate students (N = 82) learned about microbiology with Crystal Island, a game-based learning environment (GBLE), which required participants to interact with instructional materials (i.e., books and research articles, non-player character [NPC] dialogue, posters) spread throughout the game. Participants were randomly assigned to one of two conditions: full agency, where they had complete control over their actions, and partial agency, where they were required to complete an ordered play-through of Crystal Island. As participants learned with Crystal Island, log-file and eye-tracking time series data were collected to pinpoint instances when participants interacted with instructional materials. Hierarchical linear growth models indicated relationships between eye gaze dwell time and (1) the type of representation a learner gathered information from (i.e., large sections of text, poster, or dialogue); (2) the ability of the learner to distinguish relevant from irrelevant information; (3) learning gains; and (4) agency. Auto-recurrence quantification analysis (aRQA) revealed the degree to which repetitive sequences of interactions with instructional material were random or predictable. Through hierarchical modeling, analyses suggested that greater dwell times and learning gains were associated with more predictable sequences of interaction with instructional materials. Results from hierarchical clustering found that participants with restricted agency and more recurrent action sequences had greater learning gains. Implications are provided for how learning unfolds over learners' time in game using a non-linear dynamical systems analysis and the extent to which it can be supported within GBLEs to design advanced learning technologies to scaffold self-regulation during game play.

Lessons Learned and Future Directions of MetaTutor: Leveraging Multichannel Data to Scaffold Self-Regulated Learning With an Intelligent Tutoring System

“…We have also used unsupervised machine learning techniques to examine ( Lallé et al, 2018 , 2021 ; Wortha et al, 2019 ; Wiedbusch and Azevedo, 2020 ) complex eye-tracking data and facial expressions of emotions during learning with MetaTutor. We continue to use non-traditional statistical techniques, including dynamical systems modeling ( Dever et al, in press ) to examine learners’ emergent SRL behaviors, and MLAs to predict performance at the end of the learning session ( Mu et al, 2020 ; Saint et al, 2020 ; Chango et al, 2021 ; Fan et al, 2021 ). Despite our ability to continuously adapt and use contemporary analytical techniques that emerge from the computational, engineering, psychological, statistical, and data sciences, we as a field are still faced with a major barrier that continues to impact the educational effectiveness of intelligent systems such as MetaTutor.…”

“…This study explored the relationship between students' average monitoring micro-process strategy frequencies and learning gains through a personcentered approach as students interacted with MetaTutor. Using hierarchical clustering, Dever et al (2021) found that clusters differing in metacognitive monitoring process usage had a significant difference in their learning gains where students who used a greater proportion of CEs and FOKs had greater learning gains than learners who used greater MPTG strategies. These aforementioned studies demonstrate differences of when students engage in metacognitive processes during learning with MetaTutor.…”

Explicit and implicit effects of gaming content on social media on the behavior of young adults

“…Online games have grown immensely popular over the past few decades, both as recreation and for specific applications such as mental, physical, and cognitive training ( Sailer and Homner, 2020 ; Dever et al, 2022 ; Han et al, 2022 ; Aydin and Kuş, 2023 ). These games can promote relaxation by allowing participants to leave their regular duties, and experience fantastical environments ( Sailer and Homner, 2020 ; Maldonado-Murciano et al, 2022 ).…”

Early prediction of student knowledge in game‐based learning with distributed representations of assessment questions

“…The majority of studies investigating the assessment and prediction of student knowledge in game‐based learning focus on one student population with a fixed set of assessment questions that are designed for that population (eg, multiple choice questions administered after the game) (Dever et al, 2021, 2022; Henderson et al, 2020; Taub et al, 2017, Taub, Sawyer, Lester, et al, 2020). This approach assumes that the content that will be assessed will not change.…”