Levels of explicability for medical artificial intelligence: What do we normatively need and what can we technically reach?

Abstract

Definition of the problem The umbrella term “explicability” refers to the reduction of opacity of artificial intelligence (AI) systems. These efforts are challenging for medical AI applications because higher accuracy often comes at the cost of increased opacity. This entails ethical tensions because physicians and patients desire to trace how results are produced without compromising the performance of AI systems. The centrality of explicability within the informed consent process for medical AI systems compels an ethical reflection on the trade-offs. Which levels of explicability are needed to obtain informed consent when utilizing medical AI? Arguments We proceed in five steps: First, we map the terms commonly associated with explicability as described in the ethics and computer science literature, i.e., disclosure, intelligibility, interpretability, and explainability. Second, we conduct a conceptual analysis of the ethical requirements for explicability when it comes to informed consent. Third, we distinguish hurdles for explicability in terms of epistemic and explanatory opacity. Fourth, this then allows to conclude the level of explicability physicians must reach and what patients can expect. In a final step, we show how the identified levels of explicability can technically be met from the perspective of computer science. Throughout our work, we take diagnostic AI systems in radiology as an example. Conclusion We determined four levels of explicability that need to be distinguished for ethically defensible informed consent processes and showed how developers of medical AI can technically meet these requirements.

Artificial Intelligence to support ethical decision-making for incapacitated patients: a survey among German anesthesiologists and internists

“…If AI were tailored within the boundaries of clarified roles, this would allow the systems to be reliably seen as tools that help to enhance professionals’ abilities, rather than as a rival trying to render the user redundant. The aforementioned human oversight goes hand in hand with the prerequisite of transparency, which not only refers to the often mentioned explicability of the algorithms per se, but also the transparency that AI is being used at all, which is important to disclose to patients to maintain a trustful relationship [ 38 ].…”

“…Professional bodies, such as radiological associations, justify the explicability mostly as a vehicle for the principle of non-maleficence because there is a need to reduce harms inflicted by performance errors of medical AI [ 43 ]. General solutions for explainable AI in medicine are inherently interpretable models, feature visualization, prototypes, counterfactuals or feature attribution [ 38 ].…”