Non-Asimov Explanations: Regulating AI Through Transparency

Abstract

An important part of law and regulation is demanding explanations for actual and potential failures. We ask questions like: What happened (or might happen) to cause this failure? And why did (or might) it happen? These are disguised normative questions -they really ask what ought to have happened, and how the humans involved ought to have behaved. If we ask the same questions about AI systems we run into two difficulties. The first is what might be described as the 'black box'problem, which lawyers have begun to investigate. Some modern AI systems are highly complex, so that even their makers might be unable to understand their workings fully, and thus answer the what and why questions. Technologists are beginning to work on this problem, aiming to use technology to explain the workings of autonomous systems more effectively, and also to produce autonomous systems which are easier to explain.But the second difficulty is so far underexplored, and is a more important one for law and regulation. This is that the kinds of explanation required by law and regulation are not, at least at first sight, the kinds of explanation which AI systems can currently provide.To answer the normative questions, law and regulation seeks a narrative explanation, a story. Humans usually explain their  Chris Reed, Professor of Electronic Commerce Law at the Centre for Commercial Law Studies,

On the Path to the Future: Mapping the Notion of Transparency in the EU Regulatory Framework for AI

“…Humans require a narrative form of explanation which opposes the binary nature of AI systems' outputs. As noted by Reed, Grieman, and Early (2021), most citizens would not trust any AI system if they were simply told 'We cannot explain how it works, but it is really safe'. This prompted a development of an entire field of eXplainable AI (XAI) which focuses on designing tools that can enable explanations for the decisions produced by complex autonomous systems.…”