The many dimensions of algorithmic fairness in educational applications

Abstract

The issues of algorithmic fairness and bias have recently featured prominently in many publications highlighting the fact that training the algorithms for maximum performance may often result in predictions that are biased against various groups. Educational applications based on NLP and speech processing technologies often combine multiple complex machine learning algorithms and are thus vulnerable to the same sources of bias as other machine learning systems. Yet such systems can have high impact on people's lives especially when deployed as part of highstakes tests. In this paper we discuss different definitions of fairness and possible ways to apply them to educational applications. We then use simulated and real data to consider how test-takers' native language backgrounds can affect their automated scores on an English language proficiency assessment. We illustrate that total fairness may not be achievable and that different definitions of fairness may require different solutions.

Should You Fine-Tune BERT for Automated Essay Scoring?

“…Years ago, researchers suggested that demographic bias is worth checking in AES systems (Williamson et al, 2012). But years later, the field has primarily reported fairness experiments on simulated data, and shared toolkits for measuring bias, rather than results on real-world AES implementations or high-stakes data (Madnani et al, 2017;Loukina et al, 2019).…”

Algorithmic Bias in Education

“…Chouldechova (2017) comes to similar conclusions when analyzing the COMPAS dataset (Angwin et al, 2016) in light of competing fairness perspectives of calibration, predictive parity, and the balance of false predictions across groups. Educationspecific analyses have pointed out similar trade-offs in automated scoring for language proficiency exams (Loukina et al, 2019) and predictions of above-median grades in required college courses (H. .…”

“…Along with this expanded set of formalized metrics and their clear contribution to clarifying algorithmic bias has come the recognition that applying fairness measures in practice reveals its own range of obstacles and complexities. Specifically, technical obstacles to the use of fairness metrics manifest in several "impossibility" results (Chouldechova, 2017;Kleinberg et al, 2017;Berk et al, 2018;Loukina et al, 2019;, Darlington, 1971, where satisfaction of one statistical criterion of fairness makes "impossible" satisfaction of another. Kleinberg at al.…”

User-centered & Robust NLP OSS: Lessons Learned from Developing & Maintaining RSMTool

“…The goal of RSMTool is to encourage comprehensive reporting of model performance and to make it easier for stakeholders to compare different models along all necessary dimensions before model deployment. This includes not only standard agree-ment metrics, but also metrics developed within the educational measurement community and not commonly found in existing Python packages, such as measures of system performance based on test theory (Haberman, 2008;Loukina et al, 2020) as well as measures to evaluate fairness of system scores (Williamson et al, 2012;Madnani et al, 2017;Loukina et al, 2019). In this respect, our approach is similar in spirit to "Model cards" proposed by Mitchell et al (2019) or standardized data statements advocated by Bender and Friedman (2018).…”