Modeling Epistemological Principles for Bias Mitigation in AI Systems

Abstract

Artificial Intelligence (AI) has been used extensively in automatic decision making in a broad variety of scenarios, ranging from credit ratings for loans to recommendations of movies. Traditional design guidelines for AI models focus essentially on accuracy maximization, but recent work has shown that economically irrational and socially unacceptable scenarios of discrimination and unfairness are likely to arise unless these issues are explicitly addressed. This undesirable behavior has several possible sources, such as biased datasets used for training that may not be detected in black-box models. After pointing out connections between such bias of AI and the problem of induction, we focus on Popper's contributions after Hume's, which offer a logical theory of preferences. An AI model can be preferred over others on purely rational grounds after one or more attempts at refutation based on accuracy and fairness. Inspired by such epistemological principles, this paper proposes a structured approach to mitigate discrimination and unfairness caused by bias in AI systems. In the proposed computational framework, models are selected and enhanced after attempts at refutation. To illustrate our discussion, we focus on hiring decision scenarios where an AI system filters in which job applicants should go to the interview phase.

Algorithmic equity in the hiring of underrepresented IT job candidates

“…Another issue is that algorithms can obscure discrimination in ways that are unfair and unfamiliar (Barocas and Selbst, 2016). The AI software becomes more intelligent and exhibits more agency, but the predictive and decision-making processes used by algorithms are often opaque – it is difficult to explain why a particular decision was made (Dineen et al , 2004; Konradt et al , 2016; Vasconcelos et al , 2017; McCarthy et al , 2017). Also, when algorithms use non-work related data to make inferences about applicants’ age, race, religion and sex, it makes it difficult to determine if firms are adhering to federal laws that protect job applicants against discrimination (Vasconcelos et al , 2017; Valentino-DeVries, 2013).…”

“…If the historical data have an implicit bias that favors white men over Latinos, for example, then the measure of a strong candidate may have a strong correlation to race, ethnicity and gender, even if the algorithm-designer has no intention of replicating past hiring decisions that marginalize groups of people based on these categories (Barocas and Selbst, 2016). IBM researchers (Vasconcelos et al , 2017) found that, in creating these algorithms, companies model historical patterns of hiring from data describing “high-performance” employees as a basis for selecting candidates with similar profiles. Consequently, biases rooted in the traditional hiring process are reproduced and encoded in the data used to train the new systems and can have drastic impacts on human lives (O’Neil, 2016; Kilpatrick, 2016; Giang 2018).…”

Managing bias and unfairness in data for decision support: a survey of machine learning and data engineering approaches to identify and mitigate bias and unfairness within data management and analytics systems

“…For instance, German et al [58] see code reviewing as a decision process where codes from different categories of population might be more or less often accepted, Rahman et al and Bird et al [25,145] point out that bug-fix datasets are biased due to historical decisions of the engineers producing data samples. Other papers such as [16,22,24,61,80,136,165,189] reflect on how projects (data science process, creation of fairness definitions) are conducted and how unfairness is seen and might arise in general from the problem formulation perspective.…”

Master and Slave: the Dialectic of Human-Artificial Intelligence Engagement

“…By showing the outcomes of a large number of decisions made by AI that are based on demonstrated individual decisions, it can become visible and how the totality of incremental bias results in harmful treatment. Furthermore, when explicit processes are put into place to examine datasets before AI uses them to develop a decision function, data mining facilitated by AI can also help to mitigate bias if it is identified as being present in the dataset (Vasconcelos et al 2018). If using AI can help us see who we are and what we are becoming, it offers a chance to examine whether this is who we want to be.…”