Slave to the Algorithm? Why a  Right to Explanationn is Probably Not the Remedy You are Looking for

Edwards, Lilian; Veale, Michael

doi:10.2139/ssrn.2972855

Cited by 112 publications

(109 citation statements)

References 12 publications

Supporting

Mentioning

108

Contrasting

Unclassified

Order By: Relevance

“…There is significant discussion as to precisely what these provisions entail in practice regarding algorithmic decision-making, automation and profiling and whether they are adequate to address the concerns that arise from such processes (see e.g. Edwards & Veale 2017;Wachter, Mittelstadt & Floridi 2017b Most prominent of the EU initiatives has been the European Union High-Level Expert Group on Artificial Intelligence (a multi-stakeholder group of 52 experts from academia, civil society and industry) finalising its Ethics Guidelines for Trustworthy AI in April 2019 (2019a). They include 7 key, but nonexhaustive, requirements that AI systems should meet in order to be 'trustworthy'.…”

Section: European Unionmentioning

confidence: 99%

Artificial Intelligence, Governance and Ethics: Global Perspectives

Daly

Hagendorff

et al. 2019

SSRN Journal

View full text Add to dashboard Cite

Section: European Unionmentioning

confidence: 99%

Artificial Intelligence, Governance and Ethics: Global Perspectives

Daly

Hagendorff

et al. 2019

SSRN Journal

View full text Add to dashboard Cite

“…While various tools designed to help decision-makers identify and correct discrimination in data mining exist, these are aimed at data scientists implementing systems rather providing information to the individual decision-subjects affected by them [3,59,29,23,28]. The potential for local, pedagogical explanation systems to provide justice-related information, fulfilling the policy goals of transparency, accountability and fairness, has recently been noted by computer scientists and law scholars [20,65,82]. It has been suggested that organisations might rely upon these explanation facilities to fulfill legal duties to provide meaningful information about the logic of specific system outputs to affected individuals.…”

Section: Interpreting Intelligent Systemsmentioning

confidence: 99%

'It's Reducing a Human Being to a Percentage'

Binns

Kleek

Veale

et al. 2018

Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

Self Cite

298

View full text Add to dashboard Cite

Data-driven decision-making consequential to individuals raises important questions of accountability and justice. Indeed, European law provides individuals limited rights to 'meaningful information about the logic' behind significant, autonomous decisions such as loan approvals, insurance quotes, and CV filtering. We undertake three experimental studies examining people's perceptions of justice in algorithmic decision-making under different scenarios and explanation styles. Dimensions of justice previously observed in response to human decision-making appear similarly engaged in response to algorithmic decisions. Qualitative analysis identified several concerns and heuristics involved in justice perceptions including arbitrariness, generalisation, and (in)dignity. Quantitative analysis indicates that explanation styles primarily matter to justice perceptions only when subjects are exposed to multiple different styles-under repeated exposure of one style, scenario effects obscure any explanation effects. Our results suggests there may be no 'best' approach to explaining algorithmic decisions, and that reflection on their automated nature both implicates and mitigates justice dimensions.

show abstract

“…In these cases, it is unclear at what point the 'decision' is being made. Decisions might be seen in the design process, or adaptive interfaces may be seen as 'deciding' which information to provide or withold [14]. Exercise of data protection rights is different in further ways in ambient environments [13], as smart cities and ambient computing may bring significant challenges, if, for example, they are construed as part of decision-making environments.…”

Section: Meaningful Information About the Logic Of Processingmentioning

confidence: 99%

“…A key trigger condition for the automated decision-making provisions in the GDPR (art 22) [14] centres on the degree of automation of the process. Significant decisions "based solely on automated processing" require at least consent, a contract or a basis in member state law.…”

Section: Mitigating Automation Biasmentioning

confidence: 99%

Some HCI Priorities for GDPR-Compliant Machine Learning

Veale¹,

Binns²,

Kleek³

2018

Preprint

Self Cite

View full text Add to dashboard Cite

In this short paper, we consider the roles of HCI in enabling the better governance of consequential machine learning systems using the rights and obligations laid out in the recent 2016 EU General Data Protection Regulation (GDPR)-a law which involves heavy interaction with people and systems. Focussing on those areas that relate to algorithmic systems in society, we propose roles for HCI in legal contexts in relation to fairness, bias and discrimination; data protection by design; data protection impact assessments; transparency and explanations; the mitigation and understanding of automation bias; and the communication of envisaged consequences of processing.

show abstract

Slave to the Algorithm? Why a Right to Explanationn is Probably Not the Remedy You are Looking for

Cited by 112 publications

References 12 publications

Artificial Intelligence, Governance and Ethics: Global Perspectives

Artificial Intelligence, Governance and Ethics: Global Perspectives

'It's Reducing a Human Being to a Percentage'

Some HCI Priorities for GDPR-Compliant Machine Learning

Contact Info

Product

Resources

About