Integrating machine learning into item response theory for addressing the cold start problem in adaptive learning systems

Abstract

Adaptive learning systems aim to provide learning items tailored to the behavior and needs of individual learners. However, one of the outstanding challenges in adaptive item selection is that often the corresponding systems do not have information on initial ability levels of new learners entering a learning environment. Thus, the proficiency of those new learners is very difficult to be predicted. This heavily impairs the quality of personalized items' recommendation during the initial phase of the learning environment. In order to handle this issue, known as the cold-start problem, we propose a system that combines item response theory (IRT) with machine learning. Specifically, we perform ability estimation and item response prediction for new learners by integrating IRT with classification and regression trees built on learners' side information. The goal of this work is to build a learning system that incorporates IRT and machine learning into a unified framework. We compare the proposed hybrid model to alternative approaches by conducting experiments on two educational data sets. The obtained results affirmed the potential of the proposed method. In particular, the obtained results indicate that IRT combined with Random Forests provides the lowest error for the ability estimation and the highest accuracy in terms of response prediction. This way, we deduce that the employment of machine learning in combination with IRT could indeed alleviate the effect of the cold start problem in an adaptive learning environment.

Psychometrics of MOOCs: How to Measure Proficiency?

“…For instance, to solve the cold start problem Park, Joo, Cornillie, van der Maas, and Van den Noortgate (2019) combine ERS and explanatory psychometric modeling. The same problem has been addressed in a work of Pliakos and colleagues (Pliakos, et al, 2019) but using a combination of IRT models and decision tree method from machine learning. As can be seen there are several alternative solutions for the same problem can be found.…”

Alleviating the Cold Start Problem in Adaptive Learning using Data-Driven Difficulty Estimates

“…With our relatively homogeneous participant sample learning facts of fairly similar difficulty, it is perhaps unsurprising that population-level predictions were comparable to individualised predictions. The same principle likely holds for learners: the ability to make a priori assumptions about individual learners' skills becomes more relevant in a heterogeneous population, such as when the target population includes learners at different stages in their education (Klinkenberg et al 2011) or with learning disorders like dyslexia and dyscalculia (Pliakos et al 2019).…”

A Learning Analytics Theoretical Framework for STEM Education Virtual Reality Applications

“…Decision Trees, Random Forest [42,43] Assessment of the VR-supported learning material and content. Decision Trees, Random Forest, Naïve Bayes [44,45] Prediction of student's learning performance.…”