Assessing the Causal Impact of Online Instruction due to COVID-19 on Students’ Grades and its aftermath on Grade Prediction Models

Deho, Oscar Blessed; Liu, Lin; Joksimovíc, Srécko; Li, Jiuyong; Zhan, Chen; Liu, Jixue

doi:10.1145/3524458.3547232

Cited by 6 publications

(5 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is also worth mentioning that for both gender and citizenship status, the majority and minority groups have comparable success rates (i.e, 𝑃 (𝑌 = 1), see Table 2). As a matter of mere convenience and consistent with existing literature [3,9,10], we refer to the minority groups as unprivileged and majority groups as privileged. That is to say that our designation of "privilege status" does not mean the predictive models always favour the privileged group and discriminate against the unprivileged group in this study.…”

Section: Notationmentioning

confidence: 78%

“…Recently, the LA/EDM community have raised concerns about the various state-of-the-art predictive models making discriminatory decisions against students belonging to certain demographic groups [3,9,10,15,43]. In practice, given that there are many state-of-theart predictive models to choose from-each with varying degrees of predictive performance and fairness-aside their predictive performance, it is also necessary to investigate how fair a model is when deciding on which model to choose.…”

Section: Fairness Of Predictive Models In Educationmentioning

confidence: 99%

“…By analyzing this data, universities can gain valuable insights and make data-driven decisions and policies [37]. For example, by applying machine learning (ML) models, we could predict students' final course grades [9,27,40,42], classify students into groups based on those likely to pass or fail a course [18], predict students at risk of dropping out [8,11,14,41], or measure indicators of 21st century skills acquisition [24]. This has broadened the interest in both Learning Analytics (LA) and Educational Data Mining (EDM) as fields of research and practice.…”

Section: Introductionmentioning

confidence: 99%

“…Besides many apparent benefits that predictive models in education provide, researchers have in recent years raised ethical concerns regarding the fairness of these models [3,23]. In particular, many of these predictive models have been found to make discriminatory or unfair decisions against students belonging to certain demographic groups [8,9,14,35,43]. Moreover, these predictive models have been found to have varying levels of unfairness in various contexts [14,25,35].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Assessing the Fairness of Course Success Prediction Models in the Face of (Un)equal Demographic Group Distribution

Deho¹

2023

Preprint

View full text Add to dashboard Cite

In recent years, predictive models have been increasingly used by education practitioners and stakeholders to leverage actionable insights to support student success. Usually, model selection (i.e., the decision of which predictive model to use) is largely based on the predictive performance of the models. Nevertheless, it has become important to consider fairness as an integral part of the criteria for model selection. Might a model be unfair towards certain demographic groups? Might it systematically perform poorly for certain demographic groups? Indeed, prior studies affirm this. Which model out of the lot should we choose then? Additionally, prior studies suggest demographic group imbalance in the training dataset to be a source of such unfairness. If so, would the fairness of the predictive models improve if the demographic group distribution in the training dataset becomes balanced? This study seeks to answer these questions. Firstly, we analyze the fairness of 4 of the commonly used state-of-the-art models to predict course success for 3 IT courses in a large public Australian university. Specifically, we investigate if the models serve different demographic groups equally. Secondly, to address the identified unfairness—supposedly caused by the demographic group imbalance—we train the models on 3 types of \textit{balanced data} and investigate again if the unfairness was mitigated. We found that none of the predictive models was consistently fair in all 3 courses. This suggests that model selection decision should be carefully made by both the researchers and stakeholders as the per the requirement of the domain of application. Furthermore, we found that balancing demographic groups (and class labels) is not enough—albeit can be an initial step—to ensure fairness of predictive models in education. An implication of this is that sometimes, the source of unfairness may not be immediately apparent. Therefore, “blindly” attributing the unfairness to demographic group imbalance may cause the unfairness to persist even when the data becomes balanced. We hope that our findings can guide practitioners and relevant stakeholders in making well-informed decisions

show abstract

Section: Notationmentioning

confidence: 78%

Section: Fairness Of Predictive Models In Educationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Assessing the Fairness of Course Success Prediction Models in the Face of (Un)equal Demographic Group Distribution

Deho¹

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…They used multi-class Support Vector Machines (SVM) for grade prediction. Deho et al [8] used causal tree methods to analyze the factors influencing student grades during online teaching at a certain Australian university in 2020 during the pandemic. Zhang et al [9] predicted student grades by analyzing Massive Open Online Course(MOOC) learners' behavioral data using neural network algorithms and clustering algorithms.…”

Section: Introductionmentioning

confidence: 99%

Prediction of Students’ Grade by Combining Educational Knowledge Graph and Collaborative Filtering

Zhang,

Mariano,

Bringula

2024

IEEE Access

View full text Add to dashboard Cite

Traditional collaborative filtering-based grade prediction methods overly rely on students' historical grades and overlook the content correlation between courses, resulting in lower accuracy in predicting student grades. This paper proposes a grade prediction method that combines the educational domain knowledge graph with collaborative filtering, gathering course semantic information and constructing a course knowledge graph as auxiliary information for grade prediction. Through experimentation, it has been demonstrated that the integration of the educational knowledge graph and collaborative filtering in the grade prediction method uncovers more semantic relationships between courses, thereby improving the accuracy of predicting grades for related courses. The Root Mean Square Error (RMSE) and Mean Absolute Error (MAE) metrics show a decrease when compared to collaborative filtering and K-means algorithms.The method in this paper allows for more personalized learning and recommendation in the knowledge-rich field of education with semantic richness.

show abstract

Should Learning Analytics Models Include Sensitive Attributes? Explaining the Why

Deho

Joksimovíc

et al. 2023

IEEE Trans. Learning Technol.

View full text Add to dashboard Cite

Assessing the Causal Impact of Online Instruction due to COVID-19 on Students’ Grades and its aftermath on Grade Prediction Models

Cited by 6 publications

References 13 publications

Assessing the Fairness of Course Success Prediction Models in the Face of (Un)equal Demographic Group Distribution

Assessing the Fairness of Course Success Prediction Models in the Face of (Un)equal Demographic Group Distribution

Prediction of Students’ Grade by Combining Educational Knowledge Graph and Collaborative Filtering

Should Learning Analytics Models Include Sensitive Attributes? Explaining the Why

Contact Info

Product

Resources

About