With the widespread use of learning analytics (LA), ethical concerns about fairness have been raised. Research shows that LA models may be biased against students of certain demographic subgroups. Although fairness has gained significant attention in the broader machine learning (ML) community in the last decade, it is only recently that attention has been paid to fairness in LA. Furthermore, the decision on which unfairness mitigation algorithm or metric to use in a particular context remains largely unknown. On this premise, we performed a comparative evaluation of some selected unfairness mitigation algorithms regarded in the fair ML community to have shown promising results. Using a 3‐year program dropout data from an Australian university, we comparatively evaluated how the unfairness mitigation algorithms contribute to ethical LA by testing for some hypotheses across fairness and performance metrics. Interestingly, our results show how data bias does not always necessarily result in predictive bias. Perhaps not surprisingly, our test for fairness‐utility tradeoff shows how ensuring fairness does not always lead to drop in utility. Indeed, our results show that ensuring fairness might lead to enhanced utility under specific circumstances. Our findings may to some extent, guide fairness algorithm and metric selection for a given context. What is already known about this topic LA is increasingly being used to leverage actionable insights about students and drive student success. LA models have been found to make discriminatory decisions against certain student demographic subgroups—therefore, raising ethical concerns. Fairness in education is nascent. Only a few works have examined fairness in LA and consequently followed up with ensuring fair LA models. What this paper adds A juxtaposition of unfairness mitigation algorithms across the entire LA pipeline showing how they compare and how each of them contributes to fair LA. Ensuring ethical LA does not always lead to a dip in performance. Sometimes, it actually improves performance as well. Fairness in LA has only focused on some form of outcome equality, however equality of outcome may be possible only when the playing field is levelled. Implications for practice and/or policy Based on desired notion of fairness and which segment of the LA pipeline is accessible, a fairness‐minded decision maker may be able to decide which algorithm to use in order to achieve their ethical goals. LA practitioners can carefully aim for more ethical LA models without trading significant utility by selecting algorithms that find the right balance between the two objectives. Fairness enhancing technologies should be cautiously used as guides—not final decision makers. Human domain experts must be kept in the loop to handle the dynamics of transcending fair LA beyond equality to equitable LA.

No abstract

No abstract

No abstract

In recent years, predictive models have been increasingly used by education practitioners and stakeholders to leverage actionable insights to support student success. Usually, model selection (i.e., the decision of which predictive model to use) is largely based on the predictive performance of the models. Nevertheless, it has become important to consider fairness as an integral part of the criteria for model selection. Might a model be unfair towards certain demographic groups? Might it systematically perform poorly for certain demographic groups? Indeed, prior studies affirm this. Which model out of the lot should we choose then? Additionally, prior studies suggest demographic group imbalance in the training dataset to be a source of such unfairness. If so, would the fairness of the predictive models improve if the demographic group distribution in the training dataset becomes balanced? This study seeks to answer these questions. Firstly, we analyze the fairness of 4 of the commonly used state-of-the-art models to predict course success for 3 IT courses in a large public Australian university. Specifically, we investigate if the models serve different demographic groups equally. Secondly, to address the identified unfairnes-supposedly caused by the demographic group imbalance-we train the models on 3 types of balanced data and investigate again if the unfairness was mitigated. We found that none of the predictive models was consistently fair in all 3 courses. This suggests that model selection decision should be carefully made by both the researchers and stakeholders as the per the requirement of the domain of application. Furthermore, we found that balancing demographic groups (and class labels) is not enough-albeit can be an initial step-to ensure fairness of predictive models in education. An implication of this is that sometimes, the source of unfairness may not be immediately apparent. Therefore, "blindly" attributing the unfairness to demographic group imbalance may cause the unfairness to persist even when the data becomes balanced.