Reviewable Automated Decision-Making

Abstract

This paper introduces reviewability as a framework for improving the accountability of automated and algorithmic decisionmaking (ADM) involving machine learning. We draw on an understanding of ADM as a socio-technical process involving both human and technical elements, beginning before a decision is made and extending beyond the decision itself. While explanations and other model-centric mechanisms may assist some accountability concerns, they often provide insufficient information of these broader ADM processes for regulatory oversight and assessments of legal compliance. Reviewability involves breaking down the ADM process into technical and organisational elements to provide a systematic framework for determining the contextually appropriate record-keeping mechanisms to facilitate meaningful review-both of individual decisions and of the process as a whole. We argue that a reviewability framework, drawing on administrative law's approach to reviewing human decision-making, offers a practical way forward towards more a more holistic and legally-relevant form of accountability for ADM.

Out of Context: Investigating the Bias and Fairness Concerns of "Artificial Intelligence as a Service"

“…Many of these works conclude that these issues are primarily a reflection of training set deficiencies, namely the underrepresentation of certain parts of the input space [15,22] and the lack of awareness, consideration and attention of the people building these systems [94,133,142]. Consequently, a significant body of work argues for careful consideration of the social dimension in which the model will be applied, which therefore requires the use of training data that is appropriate for capturing the full diversity of the context [27,125,140]. As we argue next, however, the nature of the AI API service provision model-which aims to provide universal 'AI building blocks' that can be used and deployed in a range of customer applications, without knowing the particulars of the customer's specific usage contexts-may render these services inherently incapable of addressing the representation issue.…”

“…Apart from the role played by industry-wide regulations and standards, transparency mechanisms [17,46,128] should be embraced and implemented by developers and organisations committed to the development of AI services. Transparency which supports broader accountability processes on training sets, models and internal processes behind these services, as well as in the widersense of acknowledgement of known-limitations, past failures, and lessons learnt [27,128] can help move AIaaS from the opaque, unknown and potentially harmful 'black-boxes' of today, to a more understandable, accepting of their own limitations, building blocks for affordable and widely accessible AI systems of the future.…”

“…decision provenance [136]), and broader, holistic socio-technical system reviews (e.g. reviewability [27] and traceability [84]).…”

“…There may therefore be a role for regulators in defining standards and ensuring consistency in auditing practices so as to assist with facilitating oversight, particularly those cognisant of the 'supply chain' that AIaaS inherently entails, and the nature of the relationships of the actors within that. Indeed, such activities also presents an opportunity for the HCI community, leveraging and extending work on how to provide means and tooling for meaningful transparency to support accountability [11,27,46,62,107].…”

“Hey SyRI, tell me about algorithmic accountability”: Lessons from a landmark case

“…"Algorithmic accountability" became the rallying term under which this renewed interest was articulated (e.g., Diakopoulos, 2015), and particularly the accountability model of public administration scholar and political Mark Bovens (Bovens, 2007b(Bovens, , 2007a(Bovens, , 2010 became dominant in the field (Cooper et al, 2022) after a literature review using it (Wieringa, 2020), though there are also other takes on accountability that is being explored (Kacianka and Pretschner, 2021). One of the difficulties that academics face is how to operationalize and make accountability in practice (e.g., Cobbe et al, 2021;Kroll, 2021). This paper adds to this strand of research through the analysis of accountability practices around a real-world case.…”