Algorithmic audits of algorithms, and the law

Abstract

Algorithmic decision making is now widespread, ranging from health care allocation to more common actions such as recommendation or information ranking. The aim to audit these algorithms has grown alongside. In this paper, we focus on external audits that are conducted by interacting with the user side of the target algorithm, hence considered as a black box. Yet, the legal framework in which these audits take place is mostly ambiguous to researchers developing them: on the one hand, the legal value of the audit outcome is uncertain; on the other hand the auditors' rights and obligations are unclear.The contribution of this paper is to articulate two canonical audit forms to law, to shed light on these aspects:• the first audit form (we coin the Bobby audit form) checks a predicate against the algorithm, while the second (Sherlock) is more loose and opens up to multiple investigations. We find that: Bobby audits are more amenable to prosecution, yet are delicate as operating on real user data. This can lead to reject by a court (notion of admissibility). Sherlock audits craft data for their operation, most notably to build surrogates of the audited algorithm. It is mostly used for acts for whistleblowing, as even if accepted as a proof, the evidential value will be low in practice.• these two forms require the prior respect of a proper right to audit, granted by law or by the platform being audited; otherwise the auditor will be also prone to prosecutions regardless of the audit outcome. This article thus highlights the relation of current audits with law, in order to structure the growing field of algorithm auditing.Society rules through law, so that law is supposed to hold service providers and their algorithms accountable. In particular, decision-making algorithms are now widespread [6]. They directly face users, and govern large portions of our lives (from apparently subtle decisions such as recommendations, to more life changing ones such criminal justice or health care allocation [14]). The legal perspective on algorithms, especially of online platforms, is evolving to strong legal frameworks. For example, at an european level, in order to protect the fundamental rights of european residents 1 , or to frame artificial intelligence systems 2 explicitly shows the willingness to better regulate algorithms.The IT perspective and the nascent field of audits. As computer scientists and engineers, we are used to design and develop algorithms that process information and that can have an important impact on society [11,19]. For fine tuning these, developers operate a controlled feedback loop on data fed as inputs to the algorithm, and the corresponding algorithm results (output accuracy for instance).Considering an exterior viewpoint (the viewpoint of users or regulators), that observes or audits the behavior of remote algorithms is less frequent. A so called black box approach on algorithms can be dated back to Moore's tests black box automata in 1956 [20]. Relatively recent and sporadic works instead placed this ...