A Survey on Bias and Fairness in Machine Learning

Abstract

With the widespread use of artificial intelligence (AI) systems and applications in our everyday lives, accounting for fairness has gained significant importance in designing and engineering of such systems. AI systems can be used in many sensitive environments to make important and life-changing decisions; thus, it is crucial to ensure that these decisions do not reflect discriminatory behavior toward certain groups or populations. More recently some work has been developed in traditional machine learning and deep learning that address such challenges in different subdomains. With the commercialization of these systems, researchers are becoming more aware of the biases that these applications can contain and are attempting to address them. In this survey, we investigated different real-world applications that have shown biases in various ways, and we listed different sources of biases that can affect AI applications. We then created a taxonomy for fairness definitions that machine learning researchers have defined to avoid the existing bias in AI systems. In addition to that, we examined different domains and subdomains in AI showing what researchers have observed with regard to unfair outcomes in the state-of-the-art methods and ways they have tried to address them. There are still many future directions and solutions that can be taken to mitigate the problem of bias in AI systems. We are hoping that this survey will motivate researchers to tackle these issues in the near future by observing existing work in their respective fields.

Putting AI ethics to work: are the tools fit for purpose?

“…We also have not set out to provide an exhaustive review of the computational techniques in the AI/ML research to address ethical issues like fairness and explainability, for this, see for example [102,103]. The system is tested Deployment…”

The politics and reciprocal (re)configuration of accountability and fairness in data-driven education

“…Trained on historical data, ML algorithms may infer (multiple) proxies for legally protected or otherwise sensitive attributes (e.g., 'race', gender, or socio-economic status), consequently introducing disparities into algorithmic predictions (Baker & Hawn [2021]; Mehrabi et al [2021]). Due to unrepresentative sampling and other technical design flaws, bias against specific groups in outcomes may occur without explicit use of protected or sensitive attributes as model features.…”

“…Secondly, by indicating deviations from the ideal (i.e., unwanted bias), they inform decision-makers about the need for interventions. Existing methods can mitigate identified biases by transforming the composition of the data used to train the algorithm, by adjusting the training process or the algorithm, or by balancing the output distribution (see Mehrabi et al [2021]). Open-source toolkits (e.g., Aequitas 2 and AI Fairness 360 3 ) enable developers of ML systems to evaluate their pre-trained models against a variety of fairness definitions, aiding in efforts to mitigate bias.…”

Natural language processing as a technique for conducting text‐based research

“…This realisation is critical to consider in applied settings. As with many big data problems, such as training facial recognition technology, developing predictive models based on language data can be problematic when the training set is not representative (e.g., Amorim et al, 2018;Mehrabi et al, 2021). For instance, when NLP is used in the context of automated essay scoring it can lead to bias if not trained on a dataset that is representative of the entire community being assessed.…”