Personalized and Automated Feedback in Summative Assessment Using Recommender Systems

Abstract

In this study we explore the use of recommender systems as a means of providing automated and personalized feedback to students following summative assessment. The intended feedback is a personalized set of test questions (items) for each student that they could benefit from practicing with. Recommended items can be beneficial for students as they can support their learning process by targeting specific gaps in their knowledge, especially when there is little time to get feedback from instructors. The items are recommended using several commonly used recommender system algorithms, and are based on the students' scores in a summative assessment. The results show that in the context of the Dutch secondary education final examinations, item recommendations can be made to students with an acceptable level of model performance. Furthermore, it does not take a computationally complex model to do so: a simple baseline model which takes into account global, student-specific, and item-specific averages obtained similar performance to more complex models. Overall, we conclude that recommender systems are a promising tool for helping students in their learning process by combining multiple data sources and new methodologies, without putting additional strain on their instructors.

Learning Analytics and Computerized Formative Assessments: An Application of Dijkstra’s Shortest Path Algorithm for Personalized Test Scheduling

“…To enhance the delivery and use of educational assessments in the classroom, de Schipper et al [23] built and implemented a recommender system that provided automated and personalized feedback to secondary school students in the Netherlands. Using techniques such as singular value decomposition and collaborative filtering, the system recommended a set of personalized practice questions to students following a high-stakes summative assessment.…”

Item Selection With Collaborative Filtering in On-The-Fly Multistage Adaptive Testing

“…Furthermore, Bergner et al (2012) formalized the relationship between IRT and CF by using the CF algorithm to estimate “difficulty-like” and “discrimination-like” parameters. Other studies applied CF methods to summative and formative assessments to provide students with personalized feedback ( de Schipper et al, 2021 ) and generate personalized test administration schedules ( Bulut et al, 2020 ; Shin & Bulut, 2021 ). These studies demonstrated the utility of the CF algorithm as a psychometric method and highlighted its main strength of finding the most suitable items efficiently.…”