Responsibility &amp; Machine Learning: Part of a Process

Singh, Jatinder; Walden, Ian; Crowcroft, Jon; Bacon, Jean

doi:10.2139/ssrn.2860048

Cited by 16 publications

(9 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Once a decision is made, it is generally important to have information of the subsequent and follow-on steps, such as quality assurance processes (and their outcomes) for a decision, as well mechanisms that review sets of historical decisions. This might include checking for model skew, or any inappropriate discrimination or other behaviour that may be manifesting [67]. Moreover, once a decision is made (and 'finalised'), generally relevant are records and logs about any actions taken to give effect to the decision.…”

Section: Consequencesmentioning

confidence: 99%

Reviewable Automated Decision-Making

Cobbe

Lee

Singh

2021

Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency

Self Cite

View full text Add to dashboard Cite

This paper introduces reviewability as a framework for improving the accountability of automated and algorithmic decisionmaking (ADM) involving machine learning. We draw on an understanding of ADM as a socio-technical process involving both human and technical elements, beginning before a decision is made and extending beyond the decision itself. While explanations and other model-centric mechanisms may assist some accountability concerns, they often provide insufficient information of these broader ADM processes for regulatory oversight and assessments of legal compliance. Reviewability involves breaking down the ADM process into technical and organisational elements to provide a systematic framework for determining the contextually appropriate record-keeping mechanisms to facilitate meaningful review-both of individual decisions and of the process as a whole. We argue that a reviewability framework, drawing on administrative law's approach to reviewing human decision-making, offers a practical way forward towards more a more holistic and legally-relevant form of accountability for ADM.

show abstract

Section: Consequencesmentioning

confidence: 99%

Reviewable Automated Decision-Making

Cobbe

Lee

Singh

2021

Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency

Self Cite

View full text Add to dashboard Cite

show abstract

“…The EU's General Data Protection Regulation [7] (GDPR), for example, obliges data controllers (i.e. the entities responsible for determining the means and purposes of processing personal data) 2 and data processors 3 to be able to demonstrate compliance with its various requirements, including the data processing principles 4 if personal data is being processed. While the GDPR applies in the EU, similar data protection frameworks exist in many other countries (with the US currently a notable outlier, though changes may be afoot) [8].…”

Section: A the Legal Impetusmentioning

confidence: 99%

“…Where data controllers are processing personal data which they have obtained from elsewhere as part of this kind of 2 GDPR, Art 4 (7). 3 GDPR, Art 4(8). 4 GDPR, Art 5.…”

Section: A the Legal Impetusmentioning

confidence: 99%

“…From a technical perspective, a dominant research focus is on 'algorithmic accountability' [2], where much discussion concerns issues of fairness, transparency, and explainability particularly regarding machine learning (ML) and the specifics of the decision-making elements. 1 Generally less considered are the challenges associated with technology's broader operational contexts [3]; often comprising several different technical components, perhaps managed by different entities (organisations), which come together to realise particular functionality. For example, it is already common for data to flow from users via a mobile app to the app's provider, then potentially on to other third-parties (e.g.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Decision Provenance: Harnessing Data Flow for Accountable Systems

2019

Self Cite

View full text Add to dashboard Cite

Demand is growing for more accountability regarding the technological systems that increasingly occupy our world. However, the complexity of many of these systems -often systems-of-systems -poses accountability challenges. A key reason for this is because the details and nature of the information flows that interconnect and drive systems, which often occur across technical and organisational boundaries, tend to be invisible or opaque. This paper argues that data provenance methods show much promise as a technical means for increasing the transparency of these interconnected systems. Specifically, given the concerns regarding ever-increasing levels of automated and algorithmic decision-making, and so-called 'algorithmic systems' in general, we propose decision provenance as a concept showing much promise. Decision provenance entails using provenance methods to provide information exposing decision pipelines: chains of inputs to, the nature of, and the flow-on effects from the decisions and actions taken (at design and run-time) throughout systems. This paper introduces the concept of decision provenance, and takes an interdisciplinary (tech-legal) exploration into its potential for assisting accountability in algorithmic systems. We argue that decision provenance can help facilitate oversight, audit, compliance, risk mitigation, and user empowerment, and we also indicate the implementation considerations and areas for research necessary for realising its vision. More generally, we make the case that considerations of data flow, and systems more broadly, are important to discussions of accountability, and complement the considerable attention already given to algorithmic specifics.

show abstract

“…The following questions are useful when thinking about what an AI might be useful for: 4 What’s the interpretability model? Can results be reproduced?…”

Section: Main Textmentioning

confidence: 99%