Proceed with Caution

Zimmermann, Annette; Lee‐Stronach, Chad

doi:10.1017/can.2021.17

Cited by 22 publications

(10 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…At least since the publication of ProPublica`s assessment of the COMPAS algorithm (Angwin et al, 2016) -used by many states in the US to inform pre-trial decisions and the study on skin-type bias by Buolamwini and Gebru (2018), there have been growing concerns that the deployment of ML algorithms used to make/inform consequential decisions reinforces structural inequalities. In turn, philosophers, computer scientists, and researchers from cognate fields are increasingly paying attention to the mechanisms through which ML algorithms disadvantage certain social groups (Barocas et al, 2019;Fazelpour & Danks, 2021;Zimmermann & Lee-Stronach, 2021).…”

Section: Case Study: Mitigating Bias In Pain Diagnosismentioning

confidence: 99%

Enabling Fairness in Healthcare Through Machine Learning

Grote

Keeling

2022

Ethics Inf Technol

View full text Add to dashboard Cite

The use of machine learning systems for decision-support in healthcare may exacerbate health inequalities. However, recent work suggests that algorithms trained on sufficiently diverse datasets could in principle combat health inequalities. One concern about these algorithms is that their performance for patients in traditionally disadvantaged groups exceeds their performance for patients in traditionally advantaged groups. This renders the algorithmic decisions unfair relative to the standard fairness metrics in machine learning. In this paper, we defend the permissible use of affirmative algorithms; that is, algorithms trained on diverse datasets that perform better for traditionally disadvantaged groups. Whilst such algorithmic decisions may be unfair, the fairness of algorithmic decisions is not the appropriate locus of moral evaluation. What matters is the fairness of final decisions, such as diagnoses, resulting from collaboration between clinicians and algorithms. We argue that affirmative algorithms can permissibly be deployed provided the resultant final decisions are fair.

show abstract

Section: Case Study: Mitigating Bias In Pain Diagnosismentioning

confidence: 99%

Enabling Fairness in Healthcare Through Machine Learning

Grote

Keeling

2022

Ethics Inf Technol

View full text Add to dashboard Cite

show abstract

“…While a thorough engagement with the substantial literature on the nuances of distributive justice is beyond the scope of this paper, it suffices to say that most fairness constraints and metrics are deeply rooted in the paradigm of distributive justice, broadly construed. Except for a few proposals for taking procedural fairness seriously [24,29,56], algorithmic fairness is firmly embedded in the distributive paradigm.…”

Section: Algorithmic Fairness: Distributive Approachmentioning

confidence: 99%

“…Hoffmann [29] draws heavily on legal studies in discussing the limits of the goods included in the prevailing paradigm of distributive justice in the literature. Zimmermann and Lee-Stronach [56] draw attention to the relevance of procedural justice to algorithmic systems and argue that reliance on algorithmic systems is procedurally unjust in contexts involving background conditions of structural injustice. The arguments presented in this paper are novel in that Grgić-Hlača et al [24] does not engage with the literature on structural injustice.…”

Section: Algorithmic Fairness and Structural Injusticementioning

confidence: 99%

“…The arguments presented in this paper are novel in that Grgić-Hlača et al [24] does not engage with the literature on structural injustice. Moreover, while Hoffmann [29] and Zimmermann and Lee-Stronach [56] briefly quote Young's work on structural injustice, they do not explicitly engage with her account for correcting the limitations of fairness metrics.…”

Section: Algorithmic Fairness and Structural Injusticementioning

confidence: 99%

See 1 more Smart Citation

Algorithmic Fairness and Structural Injustice: Insights from Feminist Political Philosophy

Kasirzadeh

2022

Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society

View full text Add to dashboard Cite

Data-driven predictive algorithms are widely used to automate and guide high-stake decision making such as bail and parole recommendation, medical resource distribution, and mortgage allocation. Nevertheless, harmful outcomes biased against vulnerable groups have been reported. The growing research field known as 'algorithmic fairness' aims to mitigate these harmful biases. Its primary methodology consists in proposing mathematical metrics to address the social harms resulting from an algorithm's biased outputs.The metrics are typically motivated by -or substantively rooted in -ideals of distributive justice, as formulated by political and legal philosophers. The perspectives of feminist political philosophers on social justice, by contrast, have been largely neglected. Some feminist philosophers have criticized the paradigm of distributive justice and have proposed corrective amendments to surmount its limitations. The present paper brings some key insights of feminist political philosophy to algorithmic fairness. The paper has three goals. First, I show that algorithmic fairness does not accommodate structural injustices in its current scope. Second, I defend the relevance of structural injustices -as pioneered in the contemporary philosophical literature by Iris Marion Young -to algorithmic fairness. Third, I take some steps in developing the paradigm of 'responsible algorithmic fairness' to correct for errors in the current scope and implementation of algorithmic fairness. CCS Concepts: • Computer systems organization → Embedded systems; • Computing methodologies → Philosophical/theoretical foundations of artificial intelligence.

show abstract

“…The use of predictive machine learning algorithms (henceforth ML algorithms) to take decisions or inform a decision-making process in both public and private settings can already be observed and promises to be increasingly common. They are used to decide who should be promoted or fired, who should get a loan or an insurance premium (and at what cost), what publications appear on your social media feed [47,49] or even to map crime hot spots and to try and predict the risk of recidivism of past offenders [66]. 1 When compared to human decision-makers, ML algorithms could, at least theoretically, present certain advantages, especially when it comes to issues of discrimination.…”

Section: Introductionmentioning

confidence: 99%

AI’s fairness problem: understanding wrongful discrimination in the context of automated decision-making

Cossette-Lefebvre

Maclure

2022

AI Ethics

View full text Add to dashboard Cite

The use of predictive machine learning algorithms is increasingly common to guide or even take decisions in both public and private settings. Their use is touted by some as a potentially useful method to avoid discriminatory decisions since they are, allegedly, neutral, objective, and can be evaluated in ways no human decisions can. By (fully or partly) outsourcing a decision process to an algorithm, it should allow human organizations to clearly define the parameters of the decision and to, in principle, remove human biases. Yet, in practice, the use of algorithms can still be the source of wrongful discriminatory decisions based on at least three of their features: the data-mining process and the categorizations they rely on can reconduct human biases, their automaticity and predictive design can lead them to rely on wrongful generalizations, and their opaque nature is at odds with democratic requirements. We highlight that the two latter aspects of algorithms and their significance for discrimination are too often overlooked in contemporary literature. Though these problems are not all insurmountable, we argue that it is necessary to clearly define the conditions under which a machine learning decision tool can be used. We identify and propose three main guidelines to properly constrain the deployment of machine learning algorithms in society: algorithms should be vetted to ensure that they do not unduly affect historically marginalized groups; they should not systematically override or replace human decision-making processes; and the decision reached using an algorithm should always be explainable and justifiable.

show abstract

Proceed with Caution

Cited by 22 publications

References 36 publications

Enabling Fairness in Healthcare Through Machine Learning

Enabling Fairness in Healthcare Through Machine Learning

Algorithmic Fairness and Structural Injustice: Insights from Feminist Political Philosophy

AI’s fairness problem: understanding wrongful discrimination in the context of automated decision-making

Contact Info

Product

Resources

About