Temporal Assessment of Self-Regulated Learning by Mining Students’ Think-Aloud Protocols

Abstract

It has been widely theorized and empirically proven that self-regulated learning (SRL) is related to more desired learning outcomes, e.g., higher performance in transfer tests. Research has shifted to understanding the role of SRL during learning, such as the strategies and learning activities, learners employ and engage in the different SRL phases, which contribute to learning achievement. From a methodological perspective, measuring SRL using think-aloud data has been shown to be more insightful than self-report surveys as it helps better in determining the link between SRL activities and learning achievements. Educational process mining on the basis of think-aloud data enables a deeper understanding and more fine-grained analyses of SRL processes. Although students’ SRL is highly contextualized, there are consistent findings of the link between SRL activities and learning outcomes pointing to some consistency of the processes that support learning. However, past studies have utilized differing approaches which make generalization of findings between studies investigating the unfolding of SRL processes during learning a challenge. In the present study with 29 university students, we measured SRL via concurrent think-aloud protocols in a pre-post design using a similar approach from a previous study in an online learning environment during a 45-min learning session, where students learned about three topics and wrote an essay. Results revealed significant learning gain and replication of links between SRL activities and transfer performance, similar to past research. Additionally, temporal structures of successful and less successful students indicated meaningful differences associated with both theoretical assumptions and past research findings. In conclusion, extending prior research by exploring SRL patterns in an online learning setting provides insights to the replicability of previous findings from online learning settings and new findings show that it is important not only to focus on the repertoire of SRL strategies but also on how and when they are used.

Influencing factors of self-regulated learning of medical-related students in a traditional Chinese medical university: a cross-sectional study

“…Moreover, specific learning strategies should also be taught, such as teaching students to plan, monitor, and regulate their learning behavior, training them to restate knowledge, refine and organize them. At the same time, teaching them how to manage the learning time and environment, and asking for help during difficulties, such as Mining Students' Think-aloud Protocols, is applied to evaluate the SRL time [64]. Meanwhile, students should be taught to regulate themselves in learning.…”

A multilevel framework for assessing self‐regulated learning in school contexts: Innovations, challenges, and future directions

“…Like SRL microanalysis, think‐aloud protocols have been used to assess student SRL across various academic areas and activities. Think‐aloud protocols have been used with diverse samples across grade levels (e.g., primary and secondary students, undergraduate students, medical students), languages (e.g., English, Flemish, German), and learning domains (e.g., CBLEs; Azevedo, 2005; Fox, 2009; Heirweg et al, 2020; Lajoie et al, 2021; Lim et al, 2021; Vandevelde et al, 2017). There is extensive evidence to support claims about its convergent and criterion‐related validity, as reflected by its strong correlations with other SRL event measures (e.g., behavioral traces) and achievement outcomes across reading, science, mathematics, and history content areas (Azevedo et al, 2004; Fox, 2009; Greene et al, 2018; Muis, 2008; Rovers et al, 2019).…”

“Replacing teachers? Doubt it.” Practitioners' views on adaptive learning technologies' impact on the teaching profession

“…(Lodge et al, 2018) and adjust external regulation based on insights gained from data (Molenaar et al, 2019). Developing such a technology requires collecting and analyzing data about a learner's self-regulated learning behavior (Lim et al, 2021;Fan et al, 2022a,b) and using the generated insights to provide personalized feedback for the learner (Pardo et al, 2019(Pardo et al, , 2022. Currently, such systems are at their preliminary stages, and most solutions rely on a human teacher who is in control (Molenaar, 2021).…”