Toward learning progression analytics — Developing learning environments for the automated analysis of learning using evidence centered design

Abstract

National educational standards stress the importance of science and mathematics learning for today’s students. However, across disciplines, students frequently struggle to meet learning goals about core concepts like energy. Digital learning environments enhanced with artificial intelligence hold the promise to address this issue by providing individualized instruction and support for students at scale. Scaffolding and feedback, for example, are both most effective when tailored to students’ needs. Providing individualized instruction requires continuous assessment of students’ individual knowledge, abilities, and skills in a way that is meaningful for providing tailored support and planning further instruction. While continuously assessing individual students’ science and mathematics learning is challenging, intelligent tutoring systems show that it is feasible in principle. However, the learning environments in intelligent tutoring systems are typically not compatible with the vision of how effective K-12 science and mathematics learning looks like. This leads to the challenge of designing digital learning environments that allow for both – meaningful science and mathematics learning and the reliable and valid assessment of individual students’ learning. Today, digital devices such as tablets, laptops, or digital measurement systems increasingly enter science and mathematics classrooms. In consequence, students’ learning increasingly produces rich product and process data. Learning Analytics techniques can help to automatically analyze this data in order to obtain insights about individual students’ learning, drawing on general theories of learning and relative to established domain specific models of learning, i.e., learning progressions. We call this approach Learning Progression Analytics (LPA). In this manuscript, building of evidence-centered design (ECD), we develop a framework to guide the development of learning environments that provide meaningful learning activities and data for the automated analysis of individual students’ learning – the basis for LPA and scaling individualized instruction with artificial intelligence.

A Study on the Curriculum of the Graduate School of Department of Education Consulting in South Korea

“…In this context, AAS can significantly enhance reflection-in-action when integrated into the teaching process. These systems provide real-time data and feedback about student learning trajectories (Kubsch et al, 2022;Moon et al, 2023), affective states (Dai & Ke, 2022), and performance (Slater & Baker, 2019). The synergy between AAS and reflective practices (in-action and on-action) creates a powerful tool for continuous improvement in teaching and student learning.…”

An application of Bayesian inference to examine student retention and attrition in the STEM classroom

“…Bolstered by the emergence of newer technological innovations such as virtual reality, augmented reality, and gamification in the classroom, digital tools continue to supplement traditional pedagogical strategies and have spurred the development of diverse and novel data sources (Huang et al, 2019;Sailer and Homner, 2020;Yang et al, 2021;Alam, 2022). Despite these advances, a major challenge that has emerged with the growth of classroom technology is how to meaningfully derive cognitive insights and inferences pertaining to student learning and performance from the plethora of data and knowledge created and contained within these systems (Van Camp et al, 2017;Chen X. et al, 2020;Musso et al, 2020;Yang et al, 2021;Kubsch et al, 2022).…”

Evolving landscape of artificial intelligence (AI) and assessment in education: A bibliometric analysis

“…Besides, Zhai et al (2021) put forth the argument that machine learning (ML) has the potential to enhance educational assessment by effectively capturing complex constructs, deriving precise inferences from intricate data, and simplifying the task of human grading. In parallel, commentaries and position papers (Kubsch et al, 2022;Li et al2023;Zhai & Nehm, 2023) have extensively deliberated on the argument presented by Zhai et al (2021). These discussions have centered around the crucial topics of equity and bias concerns, shedding light on the ethical considerations surrounding the utilization of AI in formative assessment.…”