Improving Student Modeling Through Partial Credit and Problem Difficulty

Abstract

Student modeling within intelligent tutoring systems is a task largely driven by binary models that predict student knowledge or next problem correctness (i.e., Knowledge Tracing (KT)). However, using a binary construct for student assessment often causes researchers to overlook the feedback innate to these platforms. The present study considers a novel method of tabling an algorithmically determined partial credit score and problem difficulty bin for each student's current problem to predict both binary and partial next problem correctness. This study was conducted using log files from ASSISTments, an adaptive mathematics tutor, from the 2012-2013 school year. The dataset consisted of 338,297 problem logs linked to 15,253 unique student identification numbers. Findings suggest that an efficiently tabled model considering partial credit and problem difficulty performs about as well as KT on binary predictions of next problem correctness. This method provides the groundwork for modifying KT in an attempt to optimize student modeling.

Enhancing the efficiency and reliability of group differentiation through partial credit

“…The gold standard for student modeling, Knowledge Tracing (KT), has maintained its reign for almost a quarter-century despite relying on a rudimentary sequence of correct and incorrect responses to estimate the probability of student knowledge [2]. Attempts to enrich this approach have included supplemental estimates of prior knowledge to individualize predictions to each student [9], supplemental estimates of item difficulty to individualize to each problem [10], and the implementation of flexible correctness via consideration of hint usage and attempt count [12,13,7]. Despite these excursions, popular learning systems, including the Cognitive Tutor series, still largely rely on traditional KT to inform mastery learning [4].…”

Sequencing educational content in classrooms using Bayesian knowledge tracing

“…It differs from Wang and Heffernan [20] in that it does not rely on the EM algorithm, and differs from Ostrow et al [14] in that it does not use predefined methods to determine the posterior distributions. For example, receiving a score of 66% for the question shown in Figure 2 means that 66% of the student's response is correct, while 34% of the student's response is incorrect.…”

“…The first domain was the K9 dataset described in Section 3, while the second domain was the ASSISTment data set used by Ostrow et al [14]. Table 1 shows the distribution of the questions over four years of use of the K9 system, including the number of different questions, the number of students and the number of responses.…”

“…The second domain was the ASSISTment dataset used by Ostrow et al [14], which was compiled from problem logs from the ASSISTments platform during the 2012-2013 school year. 4 Here, we used two of the BKT models (BA-SIC and KT-IDEM) using the partial credit score approach described above.…”

“…Wang and Heffernan [20] extended the model to account for partial credit scores, by assigning continuous values to the question node in the model and learning Gaussian distributions over the guess and slip parameters with fixed standard deviations. Ostrow et al [14] adapt the BKT model to handle partial credit score using a "tabling" method. They maintain a discrete probability distribution table over item correctness that depends on the partial credit scores attributed to the question and its item difficulty.…”