Transparency, Compliance, And Contestability When Code Is(n't) Law

Abstract

Both technical security mechanisms and legal processes serve as mechanisms to deal with misbehaviour according to a set of norms. While they share general similarities, there are also clear differences in how they are defined, act, and the effect they have on subjects. This paper considers the similarities and differences between both types of mechanisms as ways of dealing with misbehaviour, and where they interact with each other. Taking into consideration the idea of code as law, we discuss accountability mechanisms for code, and how they must relate to both security principles and legal principles. In particular, we identify the ability to contest norms enforced by code as an important part of accountability in this context. Based on this analysis, we make the case for transparency enhancing technologies as security mechanisms that can support legal processes, in contrast to other types of accountability mechanisms for code. We illustrate this through two examples based on recent court cases that involved Post Office in the United Kingdom and Uber in the Netherlands, and discuss some practical considerations.

Transparency Enhancing Technologies to Make Security Protocols Work for Humans

“…Even if we perfected the design of security mechanisms, designing and implementing complex systems that are entirely formally verified is currently unrealistic and would not prevent harms that occur because of a system that, operating as planned, applies harmful norms [88]. Information is routinely copied, aggregated, and analysed across networks operated by different parties, rendering strict enforcement mechanisms impractical compared to relying on accountability [203].…”

“…More generally, evaluating strict compliance with norms assumes that there are reliable norms, despite many systems operating in grey areas [88]. As systems grow larger in size, complexity, and scope of applications that impact people's lives, the ability to evaluate systems is therefore very important, not only for auditors or regulators but also for users who may change how they interact with the system when given information about it [65].…”

“…The external mechanisms allow users of transparency to relate to each other and take action (e.g., contest the system [88]) based on what transparency reveals.…”

“…This entails supporting processes such as public discussions about the system to which transparency is applied and, for practical accountability purposes, legal processes that resolve disputes about a system or more automated processes that similarly make it possible to contest actions taken by the system. This is a key difference between tools that evaluate the compliance of a system with preset norms e.g., the correct execution of a program, and transparency enhancing technologies which can allow the norms enforced by a program to be contested [88].…”