Leveraging complexity frameworks to refine theories of engagement: Advancing self‐regulated learning in the age of artificial intelligence

Abstract

Capturing evidence for dynamic changes in self‐regulated learning (SRL) behaviours resulting from interventions is challenging for researchers. In the current study, we identified students who were likely to do poorly in a biology course and those who were likely to do well. Then, we randomly assigned a portion of the students predicted to perform poorly to a science of learning to learn intervention where they were taught SRL study strategies. Learning outcome and log data (257 K events) were collected from n = 226 students. We used a complex systems framework to model the differences in SRL including the amount, interrelatedness, density and regularity of engagement captured in digital trace data (ie, logs). Differences were compared between students who were predicted to (1) perform poorly (control, n = 48), (2) perform poorly and received intervention (treatment, n = 95) and (3) perform well (not flagged, n = 83). Results indicated that the regularity of students' engagement was predictive of course grade, and that the intervention group exhibited increased regularity in engagement over the control group immediately after the intervention and maintained that increase over the course of the semester. We discuss the implications of these findings in relation to the future of artificial intelligence and potential uses for monitoring student learning in online environments. Practitioner notesWhat is already known about this topic Self‐regulated learning (SRL) knowledge and skills are strong predictors of postsecondary STEM student success. SRL is a dynamic, temporal process that leads to purposeful student engagement. Methods and metrics for measuring dynamic SRL behaviours in learning contexts are needed. What this paper adds A Markov process for measuring dynamic SRL processes using log data. Evidence that dynamic, interaction‐dominant aspects of SRL predict student achievement. Evidence that SRL processes can be meaningfully impacted through educational intervention. Implications for theory and practice Complexity approaches inform theory and measurement of dynamic SRL processes. Static representations of dynamic SRL processes are promising learning analytics metrics. Engineered features of LMS usage are valuable contributions to AI models.

IA generativa y ChatGPT en Educación: Un reto para la evaluación y ¿una nueva pedagogía?

“…En particular para justificar el papel que tradicionalmente se asigna a la metacognición, la autorregulación y la retroalimentación podemos citar las siguientes dos aportaciones como más significativas: Hilpert et al (2023) abordan la necesidad de participación en el aprendizaje de STEM, con una mayor necesidad de autorregulación de los estudiantes. Se basan en una extensa historia de la ciencia de la investigación del aprendizaje que utiliza datos de seguimiento digital para crear construcciones cognitivas que brindan información sobre el compromiso, las redes sociales, la comunidad y la metacognición.…”

Advancing theory in the age of artificial intelligence

“…Hilpert et al (2023) address the need for engagement in STEM learning, with an increased need in student self‐regulation. They build on an extensive history of science of learning research that uses digital track data to create cognitive constructs that provide insight into engagement, social networks, community and metacognition.…”

“…With future R&D in mind, the authors outline the key problems, consequences, AI opportunities and desired outcomes for literacy which will undoubtedly be a helpful guide as the field progresses in coming years. Hilpert et al (2023) address the need for engagement in STEM learning, with an increased need in student self-regulation. They build on an extensive history of science of learning research that uses digital track data to create cognitive constructs that provide insight into engagement, social networks, community and metacognition.…”