Estimating the minimum number of opportunities needed for all students to achieve predicted mastery

Abstract

We have conducted a study on how many opportunities are necessary, on average, for learners to achieve mastery of a skill, also called a knowledge component (KC), as defined in the Open Learning Initiative (OLI) digital courseware. The study used datasets from 74 different course instances in four topic areas comprising 3813 students and 1.2 million transactions. The analysis supports our claim that the number of opportunities to reach mastery gives us new information on both students and the development of course components. Among the conclusions are a minimum of seven opportunities are necessary for each knowledge component, more if the prior knowledge among students are uneven within a course. The number of KCs in a course increases the number of opportunities needed. The number of opportunities to reach mastery can be used to identify KCs that are outliers that may be in need of better explanations or further instruction.

Reduced Learning Time with Maintained Learning Outcomes

“…There were lecture slides available and a structure for the course, but no formative questions. These formative questions with feedback are very time consuming to construct, and we did not reach the target of a minimum of eight questions per skill [2], but we are working on time saving solutions for that [9]. It should be noted that the SDA instances have only around 35 students, and the return on investment would likely be more significant in a larger course, as noted by House et al (2018).…”

“…Otherwise, there is something wrong. It could be the learning material, the question, the answering alternatives, the feedback, etc, but the important thing here is that the system gives feedback to the course developers on where something is wrong, so that it can be improved (for a more elaborate explanation on OLI, see [30]; the underlying learning model, [17]; analytics included in the learning dashboard, [22]; and learning curves, [2]).…”

“…The important thing here is not that students answered wrong, but rather what they learned by answering wrong and reading the feedback. This can be challenging to detect, but behind the scenes of the OLI environment there is a machine learning component [2] that can detect this and predict the likelihood of the student mastering a skill, which is available for students and teachers.…”

Efforts in Europe for Data-Driven Improvement of Education – A Review of Learning Analytics Research in Seven Countries

“…Other examples of research conducted by Swedish researchers include the exploration of multimodal learning analytics [66], and prediction of students' mastery of skills [67].…”

“…We are only aware of one project at the K-12 level, which focused on 21st century skills and collaborative problem-solving of students in secondary school [68]. In this project, which was funded by Swedish Association of Local Authorities and Regions (SKL), researchers aimed to assess collaborative problem-solving skills in technology-enhanced environments.…”