A red button does it all: A notification badge nudges students to close knowledge gaps in intelligent tutoring systems

Abstract

Background: Students who learn with adaptive intelligent tutoring systems (ITS) are often provided with additional learning materials to close their knowledge gaps when the system recognizes difficulties in a topic. Students may be nudged with notification badges to close their knowledge gaps. Aim: This study evaluated the effectiveness of implementing a small red button as a notification badge to close students’ knowledge gaps. Additionally, the association between closing knowledge gaps and students’ overall performance was investigated while controlling for relevant factors (e.g., students’ usage behavior, adaptive system features, and teachers’ assignment types).Samples: We considered 105,081 students (in 6,668 classes working on ~25 million problems) who used an ITS for learning mathematics.Methods: We employed hierarchical linear regression analysis to evaluate whether students were more likely to close their knowledge gaps in a period in which notification badges were implemented in the ITS, as compared to a period without notification badges. Moreover, we used psychological network analysis to evaluate whether closing knowledge gaps was associated with students’ performance, relative to other relevant factors.Results: Students closed 28% of their knowledge gaps when they were nudged with a notification badge but only closed 5% when they were not nudged. Closing knowledge gaps predicted better student performance and was a more important predictor than students’ usage behavior, other adaptive system features, and teachers’ assignment types.Conclusions: Our study highlights the relevance of adaptive nudges within ITS for closing knowledge gaps and thus improving learning.