Risk and Uncertainty Communication in Deployed AI-based Clinical Decision Support Systems: A scoping review

Abstract

Clinical decision support systems (CDSS) employing data-driven technology such as artificial intelligence, machine- and statistical-learning are increasingly deployed in healthcare settings. These systems often provide clinicians with diagnostic, prognostic, or risk scores modelled from curated patient-level data and frequently involve iterative and non-deterministic optimisation of flexible, parameterised models. All of these data and algorithms have uncertainties associated with them that should be taken into account when used to support clinical decisions at the patient level. This scoping review aims to describe the literature on how deployed data-driven CDSSs present information about uncertainty to their intended users. We describe common clinical applications of CDSSs, characterise the decisions that are being supported, and examine how the CDSS provides outputs to end users, including uncertainty at the individual patient level, as well as indirect measures such as CDSS performance against a reference standard. We conclude with a discussion and recommendations on how CDSS development can be improved.