An Interpretable Pipeline for Identifying At-Risk Students

Abstract

This paper introduces a novel approach to identify at-risk students with a focus on output interpretability through analyzing learning activities at a finer granularity on a weekly basis. Specifically, this approach converts the predicted output from the former weeks into meaningful probabilities to infer the predictions in the current week for maintaining the consecutiveness among learning activities. To demonstrate the efficacy of our model in identifying at-risk students, we compare the weekly AUCs and averaged performance (i.e., accuracy, precision, recall, and f1-score) over each course with the baseline models (i.e., Random Forest, Support Vector Machine, and Decision Tree), respectively. Furthermore, we adopt a Top- K metric to examine the number of at-risk students that the model is able to identify with high precision during each week. Finally, the model output is interpreted through a model-agnostic interpretation approach to support instructors to make informed recommendations for students’ learning. The experimental results demonstrate the capability and interpretability of our model in identifying at-risk students in online learning settings. In addition to that our work also provides significant implications in building accountable machine learning pipelines that can be used to automatically generated individualized learning interventions while considering fairness between different learning groups.

Is it time we get real? A systematic review of the potential of data-driven technologies to address teachers' implicit biases

“…There is not enough evidence to show that data-driven technologies help teachers make less biased decisions A core premise of research on data-driven technologies for teachers is that they enable teachers to engage in fairer and more equitable decision making and teaching practices (Angeli et al, 2017;Lameras and Arnab, 2022;Uttamchandani and Quick, 2022;Williamson and Kizilcec, 2022). Whilst much literature has demonstrated that biases can potentially be addressed algorithmically (e.g., Pei and Xing, 2021, from this review), here we question the extent to which this automatically leads to less biased decisions by teachers.…”

“…The framework can similarly be used to analyse interventions that were not designed to explicitly promote debiasing. For example, Figure 3B shows the framework as applied to Pei and Xing ( 2021 )'s intervention that help instructors identify students at risk of dropping out, through the use of machine learning techniques and complex and varied data visualizations. Researchers might consider whether, e.g., training in representations, could support instructors in making less biased decisions based on these varied visualizations.…”

“… Debiasing with data-driven technologies framework as applied to (A) the EQUIP intervention (Reinholz et al, 2020a , b ) and (B) an intervention to identify at-risk students (Pei and Xing, 2021 ). Only strategies represented by color-filled blocks are applied in these interventions.…”

Automatic Classification of Efl Learners’ Self-Reported Text Documents Along an Affective Continuum

“…These two classifiers are Support Vector Machines (Joachims, 1998) and Naïve Bayes (Chen et al, 2009). It should be noted that these kinds of classifiers were used in some previous studies related to education and computation (Liu et al, 2021;Pei & Xing, 2021). These methods are briefly explained in the next subsections.…”

Evaluating the Explainers: Black-Box Explainable Machine Learning for Student Success Prediction in MOOCs

“…[11] suggested interventions for wheel-spinning students based on Shapley values. Finally, [12] explored LIME on ensemble machine learning methods for student performance prediction, [13] integrated LIME explanations in student advising dashboards, and [14] used LIME for interpreting models identifying at-risk students.…”