An empirical study on the perceived fairness of realistic, imperfect machine learning models

Harrison, Galen; Hanson, Julia; Jacinto, Christine; Ramírez, Julio A.; Ur, Blase

doi:10.1145/3351095.3372831

Cited by 83 publications

(82 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Qualitative results confirm that the participants found the use of some factors -specifically age and gender -inappropriate in Scenario 2, so they were reluctant to believe the process was fair. Our findings confirmed our expectations and are in line with [14,23] that people who believed the decision-making process was not fair would also not trust the system's decision more than a human's decision.…”

Section: Discussionsupporting

confidence: 91%

“…Algorithms and systems should consider social and altruistic behavior in order to be considered as fair, elements that may be difficult to incorporate in mathematical modelling [8]. People tend to rate models as unfair when they consider them biased (and vice versa), and prefer human decision-making even if they consider the algorithmic model as fair or unbiased [23]. Accuracy was rated as more important than equality in [42], with demographic parity best represented people's understanding of fairness.…”

Section: Fairness In Algorithmic Decision Makingmentioning

confidence: 99%

“…The study received ethical clearance from the national ethics committee of the country where the authors' institution is operating. 4 60.6% of our respondents were male, with 47.5% in the age group of [18][19][20][21][22][23][24]35.4% between 25-32, 10.1% between 33-40, and 7.1% above 40 years old. Most of the participants (68.7%) identified themselves as a postgraduate student, and 54% of that group were Master's students.…”

Section: Participantsmentioning

confidence: 99%

See 2 more Smart Citations

I agree with the decision, but they didn't deserve this

Kasinidou

Kleanthous

Barlas

et al. 2021

Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency

View full text Add to dashboard Cite

While professionals are increasingly relying on algorithmic systems for making a decision, on some occasions, algorithmic decisions may be perceived as biased or not just. Prior work has looked into the perception of algorithmic decision-making from the user's point of view. In this work, we investigate how students in fields adjacent to algorithm development perceive algorithmic decisionmaking. Participants (N=99) were asked to rate their agreement with statements regarding six constructs that are related to facets of fairness and justice in algorithmic decision-making in three separate scenarios. Two of the three scenarios were independent of each other, while the third scenario presented three different outcomes of the same algorithmic system, demonstrating perception changes triggered by different outputs. Quantitative analysis indicates that ) 'agreeing' with a decision does not mean the person 'deserves the outcome', ) perceiving the factors used in the decision-making as 'appropriate' does not make the decision of the system 'fair' and ) perceiving a system's decision as 'not fair' is affecting the participants' 'trust' in the system. In addition, participants found proportional distribution of benefits more fair than other approaches. Qualitative analysis provides further insights into that information the participants find essential to judge and understand an algorithmic decision-making system's fairness. Finally, the level of academic education has a role to play in the perception of fairness and justice in algorithmic decision-making.

show abstract

Section: Discussionsupporting

confidence: 91%

Section: Fairness In Algorithmic Decision Makingmentioning

confidence: 99%

Section: Participantsmentioning

confidence: 99%

See 1 more Smart Citation

I agree with the decision, but they didn't deserve this

Kasinidou

Kleanthous

Barlas

et al. 2021

Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency

View full text Add to dashboard Cite

show abstract

“…As machine learning is becoming an integral part of our lives, researchers have been investigating the biases that could arise and how to mitigate them [5,44]. Work on bias mitigation looked into the interpretability and transparency of these models [27,59,60], what industry practitioners need to improve the fairness in ML systems [29,30], and the perceived fairness of biased algorithms in current practices [26,51]. In our work, we looked into how much bias and fairness in algorithms is communicated as an aspect of ML models' quality and who within teams and organizations is interested in this.…”

Section: Algorithmic Biasmentioning

confidence: 99%

How Teams Communicate about the Quality of ML Models: A Case Study at an International Technology Company

Almahmoud

DeLine

Drucker

2021

Proc. ACM Hum.-Comput. Interact.

View full text Add to dashboard Cite

Machine learning (ML) has become a crucial component in software products, either as part of the user experience or used internally by software teams. Prior studies have explored how ML is affecting development team roles beyond data scientists, including user experience designers, program managers, developers and operations engineers. However, there has been little investigation of how team members in different roles on the team communicate about ML, in particular about the quality of models. We use the general term quality to look beyond technical issues of model evaluation, such as accuracy and overfitting, to any issue affecting whether a model is suitable for use, including ethical, engineering, operations, and legal considerations. What challenges do teams face in discussing the quality of ML models? What work practices mitigate those challenges? To address these questions, we conducted a mixed-methods study at a large software company, first interviewing15 employees in a variety of roles, then surveying 168 employees to broaden our understanding. We found several challenges, including a mismatch between user-focused and model-focused notions of performance, misunderstandings about the capabilities and limitations of evolving ML technology, and difficulties in understanding concerns beyond one's own role. We found several mitigation strategies, including the use of demos during discussions to keep the team customer-focused.

show abstract

“…Next, ethical principles argue AI systems should be developed to do good or benefit someone or the society as a whole (beneficence); they should avoid doing harm to others (non-maleficence) [27,34]. Finally, rules should be established on managing conflict of interest situations within the team or when the values of the system conflict with the interests or values of the users [62,63].…”

Section: Management I Governancementioning

confidence: 99%

Artificial Intelligence Project Success Factors: Moral Decision-Making with Algorithms

Miller¹

2021

Annals of Computer Science and Information Systems

View full text Add to dashboard Cite

The algorithms implemented through artificial intelligence (AI) and big data projects are used in life-anddeath situations. While research exists to address varying aspects of moral decision-making with algorithms, the definition of project success is not readily available. Nevertheless, researchers place the burden of responsibility for ethical decisions from AI systems on the system developers. Using a systematic literature review, this research identified 71 AI project success factors in 14 groups related to moral decision-making with algorithms. It contributes to project management literature, specifically for AI projects. Project managers and sponsors can use the results during project planning and execution.

show abstract

An empirical study on the perceived fairness of realistic, imperfect machine learning models

Cited by 83 publications

References 20 publications

I agree with the decision, but they didn't deserve this

I agree with the decision, but they didn't deserve this

How Teams Communicate about the Quality of ML Models: A Case Study at an International Technology Company

Artificial Intelligence Project Success Factors: Moral Decision-Making with Algorithms

Contact Info

Product

Resources

About