Automated Data-Driven Generation of Personalized Pedagogical Interventions in Intelligent Tutoring Systems

Abstract

Intelligent tutoring systems (ITS) have been shown to be highly effective at promoting learning as compared to other computer-based instructional approaches. However, many ITS rely heavily on expert design and hand-crafted rules. This makes them difficult to build and transfer across domains and limits their potential efficacy. In this paper, we investigate how feedback in a large-scale ITS can be automatically generated in a data-driven way, and more specifically how personalization of feedback can lead to improvements in student performance outcomes. First, in this paper we propose a machine learning approach to generate personalized feedback in an automated way, which takes individual needs of students into account, while alleviating the need of expert intervention and design of hand-crafted rules. We leverage state-of-the-art machine learning and natural language processing techniques to provide students with personalized feedback using hints and Wikipedia-based explanations. Second, we demonstrate that personalized feedback leads to improved success rates at solving exercises in practice: our personalized feedback model is used in , a large-scale dialogue-based ITS with around 20,000 students launched in 2019. We present the results of experiments with students and show that the automated, data-driven, personalized feedback leads to a significant overall improvement of 22.95% in student performance outcomes and substantial improvements in the subjective evaluation of the feedback.

Inclusive Digital Education

“…Another study shows that personalised feedback improves the learning process. There, learner performance improved by 22.95% when learners received automated, personalised feedback (Kochmar et al, 2021).…”

Few-Shot Question Generation for Personalized Feedback in Intelligent Tutoring Systems

“…• Personalized Human Feedback Baseline: For every exercise, Korbit already has several hints manually crafted by course designers. To select the best hint from the ones available, the ITS uses a personalized ML model by looking at student performance and responses on the exercise [16]. This personalization is used only during hint selection, and not during hint generation itself (which is manual).…”

Deskilling of Teaching and the Case for Intelligent Tutoring Systems

“…A team at the University of Bath found a significant increase in learning after incorporating the ability to parse natural language queries and provide hints and links into their intelligent tutoring system Korbit. Their system personalizes the answers and hints based both on previous student queries and their ongoing performance in the course (Kochmar et al, 2021). A GPT-3 system from Open AI can summarize books of any length (Wu et al, 2021), and Google, Facebook, and Microsoft have developed their own document summarizing tools (Wiggers, 2021).…”