Algorithmic Bias and Risk Assessments: Lessons from Practice

Abstract

In this paper, we distinguish between different sorts of assessments of algorithmic systems, describe our process of assessing such systems for ethical risk, and share some key challenges and lessons for future algorithm assessments and audits. Given the distinctive nature and function of a third-party audit, and the uncertain and shifting regulatory landscape, we suggest that second-party assessments are currently the primary mechanisms for analyzing the social impacts of systems that incorporate artificial intelligence. We then discuss two kinds of assessments: an ethical risk assessment and a narrower, technical algorithmic bias assessment. We explain how the two assessments depend on each other, highlight the importance of situating the algorithm within its particular socio-technical context, and discuss a number of lessons and challenges for algorithm assessments and, potentially, for algorithm audits. The discussion builds on our team's experience of advising and conducting ethical risk assessments for clients across different industries in the last 4 years. Our main goal is to reflect on the key factors that are potentially ethically relevant in the use of algorithms and draw lessons for the nascent algorithm assessment and audit industry, in the hope of helping all parties minimize the risk of harm from their use.

Auditing of AI: Legal, Ethical and Technical Approaches

“…The task of studying the effectiveness and feasibility of different AI auditing procedures is thus one for academic researchers. Here, Hasan et al (2022), Felländer et al (2022) and Vetter et al (2023 all make important contributions by (i) documenting the methodological affordances and constraints of different AI auditing procedures and (ii) reflecting on the challenges auditors and industry practitioners face when attempting to design and implement AI audits in applied settings.…”

“…In Algorithmic Bias and Risk Assessments: Lessons from Practice, Hasan et al (2022) The procedure focuses on identifying and deliberating on ethical issues and tensions through the analysis of socio-technical scenarios. The authors illustrate how Z-Inspection works through realworld examples of its application to assess AI systems used in the healthcare sector and for environmental monitoring purposes.…”