Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making

Veale, Michael; Kleek, Max Van; Binns, Reuben

doi:10.1145/3173574.3174014

Cited by 310 publications

(247 citation statements)

References 66 publications

Supporting

Mentioning

242

Contrasting

Unclassified

Order By: Relevance

“…housing). In contexts in which public administration and private decision-making are being automated or augmented with algorithms, the need for occasional individualised consideration is one of the reasons for keeping a human-in-the-loop, or 'screen-level bureaucrat' on hand [1,8,12,48]. By scrutinising additional information about the individual that the algorithm does not consider, and considering alternative forms of reasoning regarding mitigating circumstances that an algorithm could not, the 'human-in-the-loop' may be able to serve the aim of individual justice.…”

Section: Aristotle's Other Maximmentioning

confidence: 99%

On the apparent conflict between individual and group fairness

Binns

2020

Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency

Self Cite

250

163

View full text Add to dashboard Cite

A distinction has been drawn in fair machine learning research between 'group' and 'individual' fairness measures. Many technical research papers assume that both are important, but conflicting, and propose ways to minimise the trade-offs between these measures. This paper argues that this apparent conflict is based on a misconception. It draws on theoretical discussions from within the fair machine learning research, and from political and legal philosophy, to argue that individual and group fairness are not fundamentally in conflict. First, it outlines accounts of egalitarian fairness which encompass plausible motivations for both group and individual fairness, thereby suggesting that there need be no conflict in principle. Second, it considers the concept of individual justice, from legal philosophy and jurisprudence which seems similar but actually contradicts the notion of individual fairness as proposed in the fair machine learning literature. The conclusion is that the apparent conflict between individual and group fairness is more of an artefact of the blunt application of fairness measures, rather than a matter of conflicting principles. In practice, this conflict may be resolved by a nuanced consideration of the sources of 'unfairness' in a particular deployment context, and the carefully justified application of measures to mitigate it.

show abstract

Section: Aristotle's Other Maximmentioning

confidence: 99%

On the apparent conflict between individual and group fairness

Binns

2020

Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency

Self Cite

250

163

View full text Add to dashboard Cite

show abstract

“…Should models be used to increase preventative measures, such as community policing, or to heighten response capacity after crimes have been reported?" (Veale, et al, 2018).…”

Section: Discussion and Way Forwardmentioning

confidence: 99%

AI governance in the public sector: Three tales from the frontiers of automated decision-making in democratic settings

Kuziemski

Misuraca

2020

Telecommunications Policy

291

145

View full text Add to dashboard Cite

The rush to understand new socio-economic contexts created by the wide adoption of AI is justified by its farranging consequences, spanning almost every walk of life. Yet, the public sector's predicament is a tragic double bind: its obligations to protect citizens from potential algorithmic harms are at odds with the temptation to increase its own efficiency -or in other words -to govern algorithms, while governing by algorithms. Whether such dual role is even possible, has been a matter of debate, the challenge stemming from algorithms' intrinsic properties, that make them distinct from other digital solutions, long embraced by the governments, create externalities that rule-based programming lacks. As the pressures to deploy automated decision making systems in the public sector become prevalent, this paper aims to examine how the use of AI in the public sector in relation to existing data governance regimes and national regulatory practices can be intensifying existing power asymmetries. To this end, investigating the legal and policy instruments asssociated with the use of AI for strenghtening the immigration process control system in Canada; "optimising" the employment services" in Poland, and personalising the digital service experience in Finland, the paper advocates for the need of a common framework to evaluate the potential impact of the use of AI in the public sector. In this regard, it discusses the specific effects of automated decision support systems on public services and the growing expectations for governments to play a more prevalent role in the digital society and to ensure that the potential of technology is harnessed, while negative effects are controlled and possibly avoided. This is of particular importance in light of the current COVID-19 emergency crisis where AI and the underpinning regulatory framework of data ecosystems, have become crucial policy issues as more and more innovations are based on large scale data collections from digital devices, and the real-time accessibility of information and services, contact and relationships between institutions and citizens could strengthen -or undermine -trust in governance systems and democracy.

show abstract

“…Since algorithms are being trained on big data, the issues that they identify have a bearing on the regulation of algorithms too. Second, as Veale et al (2018) point out, practitioners are already deploying these systems in the public sector and 'are facing immediate, value laden challenges'. They suggest that researchers should not assume that public sector practitioners are naive about 'challenges such as fairness and accountability', and urge greater engagement, based on trust, between public bodies and researchers.…”

Section: Algorithms Big Data and The Search For Public Valuementioning

confidence: 99%

Public administration, public leadership and the construction of public value in the age of the algorithm and ‘big data’

Andrews

2018

Public Administration

113

View full text Add to dashboard Cite

Public administration scholarship has to a significant degree neglected technological change. The age of the algorithm and ‘big data’ is throwing up new challenges for public leadership, which are already being confronted by public leaders in different jurisdictions. Algorithms may be perceived as presenting new kinds of ‘wicked problems’ for public authorities. The article offers a tentative overview of the kind of algorithmic challenges facing public leaders in an environment where the discursive context is shaped by corporate technology companies. Public value theory is assessed as an analytical framework to examine how public leaders are seeking to address the ethical and public value issues affecting governance and regulation, drawing on recent UK experience in particular. The article suggests that this is a fruitful area for future research.

show abstract

Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making

Cited by 310 publications

References 66 publications

On the apparent conflict between individual and group fairness

On the apparent conflict between individual and group fairness

AI governance in the public sector: Three tales from the frontiers of automated decision-making in democratic settings

Public administration, public leadership and the construction of public value in the age of the algorithm and ‘big data’

Contact Info

Product

Resources

About