'It's Reducing a Human Being to a Percentage'; Perceptions of Justice in Algorithmic Decisions

Binns, Reuben; Kleek, Max Van; Veale, Michael; Lyngs, Ulrik; Zhao, Jun; Shadbolt, Nigel

doi:10.31235/osf.io/9wqxr

Cited by 185 publications

(351 citation statements)

References 0 publications

Supporting

Mentioning

340

Contrasting

Unclassified

Order By: Relevance

“…Specifically, the interview tool we describe gathers data from interviewees, evaluates this data, and provides feedback for interviewees. It seems possible that participants might have interpreted that a computer gains decision power over a human being, which may be seen as less fair than a person being in charge of conducting high‐stake interviews and making high‐stake selection decisions (Binns et al, ; Lee, ; Ötting & Maier, ). To be clear, the issue at hand might be the lack of transparency of automated high‐stake situations in addition to a highly automated decision agent (Mittelstadt, Allo, Taddeo, Wachter, & Floridi, ).…”

Section: Discussionmentioning

confidence: 99%

“…We propose that the stakes of a context might affect reactions to highly automated tools because people might perceive that a human decision agent should be in charge of high‐stake processes and decisions instead of leaving those to highly automated tools (Binns et al, ). Additionally, setting up a high‐stake context in a virtual environment might also lead to negative feelings about the respective tool as people conclude that this kind of situation should rather be handled through an interaction with another human being maybe because of the implicit sign that there is more social recognition when another person handles an important decision (Lee, ).…”

Section: Background and Hypotheses Developmentmentioning

confidence: 99%

See 1 more Smart Citation

Highly automated job interviews: Acceptance under the influence of stakes

Langer

König

Παπαθανασίου

2019

Int J Selection Assessment

114

100

View full text Add to dashboard Cite

Technological advancements allow the automation of every part of job interviews (information acquisition, information analysis, action selection, action implementation) resulting in highly automated interviews. Efficiency advantages exist, but it is unclear how people react to such interviews (and whether reactions depend on the stakes involved). Participants (N = 123) in a 2 (highly automated, videoconference) × 2 (high‐stakes, low‐stakes situation) experiment watched and assessed videos depicting a highly automated interview for high‐stakes (selection) and low‐stakes (training) situations or an equivalent videoconference interview. Automated high‐stakes interviews led to ambiguity and less perceived controllability. Additionally, highly automated interviews diminished overall acceptance through lower social presence and fairness. To conclude, people seem to react negatively to highly automated interviews and acceptance seems to vary based on the stakes. Open Practices This study was pre‐registered on the Open Science Framework (http://osf.io/hgd5r) and on AsPredicted (https://AsPredicted.org/i52c6.pdf).

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Background and Hypotheses Developmentmentioning

confidence: 99%

Highly automated job interviews: Acceptance under the influence of stakes

Langer

König

Παπαθανασίου

2019

Int J Selection Assessment

114

100

View full text Add to dashboard Cite

show abstract

“…He has been with DeepNetwork for a little less than two years. Before this, he worked as a Product Engineer 7. Max has an undergraduate degree in information and computer science and a graduate degree in mathematics and statistics.…”

mentioning

confidence: 99%

Trust in Data Science

Passi

Jackson

2018

Proc. ACM Hum.-Comput. Interact.

134

View full text Add to dashboard Cite

The trustworthiness of data science systems in applied and real-world settings emerges from the resolution of specific tensions through situated, pragmatic, and ongoing forms of work. Drawing on research in CSCW, critical data studies, and history and sociology of science, and six months of immersive ethnographic fieldwork with a corporate data science team, we describe four common tensions in applied data science work: (un)equivocal numbers, (counter)intuitive knowledge, (in)credible data, and (in)scrutable models. We show how organizational actors establish and re-negotiate trust under messy and uncertain analytic conditions through practices of skepticism, assessment, and credibility. Highlighting the collaborative and heterogeneous nature of real-world data science, we show how the management of trust in applied corporate data science settings depends not only on pre-processing and quantification, but also on negotiation and translation. We conclude by discussing the implications of our findings for data science research and practice, both within and beyond CSCW.

show abstract

“…See [94]. 11 More generally, COMPAS has been considered for use at sentencing as well, and this is the source of the issue in the Loomis case [91]. Pre-trial sanctions loom large in the discussion of risk assessments, as reducing incarcerated populations is a major goal of criminal justice reform and in many states the bulk of the jail population is pre-sentence.…”

Section: Applying the Analytic Retrospectively To Unpack The Compas Cmentioning

confidence: 99%

“…In terms of the meaning of unfairness in the context of COMPAS, a major point of disagreement is the validity of statistical treatment, which rates individuals based only on a defined set of features as opposed to casuistically [11]. Here, jurisdictions have traded off the harms of statistical treatment for efficiency in the administration of justice, but this necessitates access to information and capability to contest erroneous decisions that is not present in COMPAS applications [91].…”

Section: Applying the Analytic Retrospectively To Unpack The Compas Cmentioning

confidence: 99%

This Thing Called Fairness

Mulligan

Kroll

Kohli

et al. 2019

Proc. ACM Hum.-Comput. Interact.

View full text Add to dashboard Cite

The explosion in the use of software in important sociotechnical systems has renewed focus on the study of the way technical constructs reflect policies, norms, and human values. This effort requires the engagement of scholars and practitioners from many disciplines. And yet, these disciplines often conceptualize the operative values very differently while referring to them using the same vocabulary. The resulting conflation of ideas confuses discussions about values in technology at disciplinary boundaries. In the service of improving this situation, this paper examines the value of shared vocabularies, analytics, and other tools that facilitate conversations about values in light of these disciplinary specific conceptualizations, the role such tools play in furthering research and practice, outlines different conceptions of "fairness"deployed in discussions about computer systems, and provides an analytic tool for interdisciplinary discussions and collaborations around the concept of fairness. We use a case study of risk assessments in criminal justice applications to both motivate our effort-describing how conflation of different concepts under the banner of "fairness" led to unproductive confusion-and illustrate the value of the fairness analytic by demonstrating how the rigorous analysis it enables can assist in identifying key areas of theoretical, political, and practical misunderstanding or disagreement, and where desired support alignment or collaboration in the absence of consensus. the terms we consider here is at the early stages of formation, now is the time to attend to the infrastructure necessary to support its development. BACKGROUNDParticularly with the rise of machine learning as a core technical tool for building computer systems and the concomitant sense that these systems were not adequately reflecting the values and goals of their designers and creators, the question of how to build values into software systems has gained significant traction in recent years. One focal point for the community has been the rise of the FAT scholarly meetings (FAT/ML, or the Workshop on Fairness, Accountability, and Transparency in Machine Learning, held annually since 2014 at a major machine learning conference and FAT * , the (now, ACM) Conference on Fairness, Accountability, and Transparency, which aims to build community outside of research with a machine learning focus. 2 In a similar vein, prior research within Computer Supported Cooperative Work (CSCW) and related fields, such as Human Computer Interaction (HCI) and Science & Technology Studies, has paid attention to the ways in which technical practices and computational artifacts of all kinds embed or promote a range of social values (e.g., [78,110,137,138,151]). Research programs developed to focus on values and technology, such as Value Sensitive Design and "values in design", include forms of both analysis to identify and critique values associated with systems, and methods for incorporating values into the processes of engineering and design [48,4...

show abstract

'It's Reducing a Human Being to a Percentage'; Perceptions of Justice in Algorithmic Decisions

Cited by 185 publications

References 0 publications

Highly automated job interviews: Acceptance under the influence of stakes

Highly automated job interviews: Acceptance under the influence of stakes

Trust in Data Science

This Thing Called Fairness

Contact Info

Product

Resources

About