Where fairness fails: data, algorithms, and the limits of antidiscrimination discourse

Hoffmann, Anna Lauren

doi:10.1080/1369118x.2019.1573912

Cited by 338 publications

(202 citation statements)

References 34 publications

Supporting

Mentioning

194

Contrasting

Unclassified

Order By: Relevance

“…Aside from these problems, the very framing of Fair-ML can also be criticised on the grounds that it centres the decision-maker and assumes the legitimacy of their power to make decisions based on algorithms, including choosing which contestable assumptions to incorporate into them [27,40]. In many cases such legitimacy is rightly challenged.…”

Section: On the Legitimacy Of Decision-makers Normative Assumptionsmentioning

confidence: 99%

On the apparent conflict between individual and group fairness

Binns

2020

Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency

250

163

View full text Add to dashboard Cite

A distinction has been drawn in fair machine learning research between 'group' and 'individual' fairness measures. Many technical research papers assume that both are important, but conflicting, and propose ways to minimise the trade-offs between these measures. This paper argues that this apparent conflict is based on a misconception. It draws on theoretical discussions from within the fair machine learning research, and from political and legal philosophy, to argue that individual and group fairness are not fundamentally in conflict. First, it outlines accounts of egalitarian fairness which encompass plausible motivations for both group and individual fairness, thereby suggesting that there need be no conflict in principle. Second, it considers the concept of individual justice, from legal philosophy and jurisprudence which seems similar but actually contradicts the notion of individual fairness as proposed in the fair machine learning literature. The conclusion is that the apparent conflict between individual and group fairness is more of an artefact of the blunt application of fairness measures, rather than a matter of conflicting principles. In practice, this conflict may be resolved by a nuanced consideration of the sources of 'unfairness' in a particular deployment context, and the carefully justified application of measures to mitigate it.

show abstract

Section: On the Legitimacy Of Decision-makers Normative Assumptionsmentioning

confidence: 99%

On the apparent conflict between individual and group fairness

Binns

2020

Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency

250

163

View full text Add to dashboard Cite

show abstract

“…Benchmark dataset curation frequently involves supplementing or highlighting data from a specific population that is underrepresented in previous datasets. Efforts to increase representation of this group can lead to tokenism and exploitation, compromise privacy, and perpetuate marginalization through population monitoring and targeted violence [22,25,35]. And the method through which companies pursue better representation can be ethically questionable.…”

Section: Tension 1: Privacy and Representationmentioning

confidence: 99%

“…how systems of power and oppression give rise to qualitatively different experiences for individuals holding multiply marginalized identities [25].…”

Section: Tensionmentioning

confidence: 99%

Saving Face

Raji

Gebru

Mitchell

et al. 2020

Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society

185

View full text Add to dashboard Cite

Although essential to revealing biased performance, well intentioned attempts at algorithmic auditing can have effects that may harm the very populations these measures are meant to protect. This concern is even more salient while auditing biometric systems such as facial recognition, where the data is sensitive and the technology is often used in ethically questionable manners. We demonstrate a set of five ethical concerns in the particular case of auditing commercial facial processing technology, highlighting additional design considerations and ethical tensions the auditor needs to be aware of so as not exacerbate or complement the harms propagated by the audited system. We go further to provide tangible illustrations of these concerns, and conclude by reflecting on what these concerns mean for the role of the algorithmic audit and the fundamental product limitations they reveal.

show abstract

“…This component illustrated a single dimension of algorithmic harmerroneous results. Conversations with our partner organizations and participants emphasized that while such demonstrations are important, they risk promoting "single axis thinking" [32] about algorithmic systems in which a focus on technical errors in a problematic technology diverts attention from the social systems that produce both the technology's inequitable effects and the narratives that justify its use. Surveillance systems that work as intended still produce undesirable effects and reproduce patterns of discrimination [35].…”

Section: Non-technical Measures Are Powerful Steps Toward Algorithmicmentioning

confidence: 99%

Toward situated interventions for algorithmic equity

Katell

Young

Dailey

et al. 2020

Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency

108

View full text Add to dashboard Cite

Research to date aimed at the fairness, accountability, and transparency of algorithmic systems has largely focused on topics such as identifying failures of current systems and on technical interventions intended to reduce bias in computational processes. Researchers have given less attention to methods that account for the social and political contexts of specific, situated technical systems at their points of use. Co-developing algorithmic accountability interventions in communities supports outcomes that are more likely to address problems in their situated context and re-center power with those most disparately affected by the harms of algorithmic systems. In this paper we report on our experiences using participatory and co-design methods for algorithmic accountability in a project called the Algorithmic Equity Toolkit. The main insights we gleaned from our experiences were: (i) many meaningful interventions toward equitable algorithmic systems are non-technical; (ii) community organizations derive the most value from localized materials as opposed to what is "scalable" beyond a particular policy context; (iii) framing harms around algorithmic bias suggests that more accurate data is the solution, at the risk of missing deeper questions about whether some technologies should be used at all. More broadly, we found that community-based methods are important inroads to addressing algorithmic harms in their situated contexts.

show abstract

Where fairness fails: data, algorithms, and the limits of antidiscrimination discourse

Cited by 338 publications

References 34 publications

On the apparent conflict between individual and group fairness

On the apparent conflict between individual and group fairness

Saving Face

Toward situated interventions for algorithmic equity

Contact Info

Product

Resources

About