The effect of adaptivity in digital learning technologies. Modelling learning efficiency using data from an educational game

Abstract

During the last decade, many governments and ed‐tech companies have demonstrated an increased interest in digital personalised learning, which resulted in a variety of often game‐like adaptive learning environments. However, there has been limited attention for the impact of these personalised learning technologies on children's learning efficiency. Does digital personalised learning, like popular claims insist, foster learning in young children? This study attempts to empirically validate the beneficial impact of adaptive learning technology by analysing log‐data from the Number Sense Game (NSG), an educational game that trains early numerical skills. In total, 81 children were randomly assigned to use either an adaptive or a non‐adaptive version of the NSG in six sessions in a three‐week period. Using a longitudinal item response model children's progress within and across sessions was modelled and compared between the two versions of the game. Regardless of the version of the NSG, children demonstrated progress within and across sessions. However, compared to the non‐adaptive NSG, the progress across sessions was stronger in the adaptive NSG. These results provide empirical evidence that adaptive learning environments can improve learning efficiency in young children. What is already known about this topic Adaptive digital technologies are able to address students’ individual needs. Pretest posttest intervention research on the effectiveness of adaptive digital technologies revealed mixed results. What this paper adds Learning effectiveness is different from learning efficiency. Learning efficiency can be operationalised using log‐data to model students’ progress over time. Adaptive digital technologies increased students’ learning efficiency compared to non‐adaptive digital technologies. Implications for practice and/or policy Log‐data can be obtained unintrusively and provides fine‐grained information about students’ learning. Researchers are encouraged to investigate the effects of adaptive digital technologies in terms of learning effectiveness and learning efficiency.

Technology‐mediated personalised learning for younger learners: Concepts, design, methods and practice

“…This evidence can be obtained by conducting controlled designs in which the effects of personalised learning environments are contrasted with non-personalised learning environments. The study of Debeer et al (2021) empirically validated the beneficial impact of an adaptive learning game by analysing the log data from an adaptive and non-adaptive version of the Number Sense Game that trains early numerical skills. The researchers used innovative approaches to model the students' progress over time, thereby giving an indication of the students' learning efficiency.…”

In‐game performance: The role of students' socio‐economic status, self‐efficacy and situational interest in an augmented reality game

“…One of the affordances of digital games is the opportunity to use log data as an objective performance measure for learning analytics (Shute et al, 2015) since it provides detailed information about students' learning (Debeer et al, 2021). Log data can be used to assess learning outcomes of students without needing assessments on paper (Alonso‐Fernández et al, 2019) and can give detailed insights into underlying attentional processes of the individual learner (Wang et al, 2022).…”

Evaluation of Learning Efficiency of Massive Open Online Courses Learners

“…In another study, Zaveri, B., et al [8] proved that learners can complete tasks quickly and recognize the usability of online learning platforms by improving interactive communication, thus enabling students to improve their learning efficiency greatly. Debeer, D., et al [9] reported that education games can improve learners' ability to adapt to learning environments and improve the learning efficiency of children.…”