Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learning

Abstract

In 1996, philosopher Helen Nissenbaum issued a clarion call concerning the erosion of accountability in society due to the ubiquitous delegation of consequential functions to computerized systems. Using the conceptual framing of moral blame, Nissenbaum described four types of barriers to accountability that computerization presented: 1) "many hands," the problem of attributing moral responsibility for outcomes caused by many moral actors; 2) "bugs," a way software developers might shrug off responsibility by suggesting software errors are unavoidable; 3) "computer as scapegoat," shifting blame to computer systems as if they were moral actors; and 4) "ownership without liability," a free pass to the tech industry to deny responsibility for the software they produce. We revisit these four barriers in relation to the recent ascendance of data-driven algorithmic systemstechnology often folded under the heading of machine learning (ML) or artificial intelligence (AI) -to uncover the new challenges for accountability that these systems present. We then look ahead to how one might construct and justify a moral, relational framework for holding responsible parties accountable, and argue that the FAccT community is uniquely well-positioned to develop such a framework to weaken the four barriers.

Adversarial Scrutiny of Evidentiary Statistical Software

“…If the defendant in question belongs to this 1% of instances, the tool will be guaranteed to fail in this case despite its seemingly impressive accuracy. 5 To formalize this intuition and ground the legal concept of adversarial scrutiny in the technical reality of statistical software, we build on conceptual advances from the literature on distributional robustness in machine learning and algorithmic fairness [15,17,31,40,56,98,121]. At its core, robust adversarial testing requires a tool to perform well on input most relevant to the defendant's case.…”

Achieving Downstream Fairness with Geometric Repair

“…The upstream developer and downstream user are often different parties; the steps detailed above might be done only by the upstream model developer, while the downstream end-user must rely solely on the model provided by the developer. Furthermore, either party may be constrained by resources, logistical barriers, or context-specific barriers, leading the downstream model user to employ a different threshold for making final classification decisions than the upstream model developer had anticipated and optimized for [19,26].…”