Explainability in medicine in an era of AI-based clinical decision support systems

Abstract

The combination of “Big Data” and Artificial Intelligence (AI) is frequently promoted as having the potential to deliver valuable health benefits when applied to medical decision-making. However, the responsible adoption of AI-based clinical decision support systems faces several challenges at both the individual and societal level. One of the features that has given rise to particular concern is the issue of explainability, since, if the way an algorithm arrived at a particular output is not known (or knowable) to a physician, this may lead to multiple challenges, including an inability to evaluate the merits of the output. This “opacity” problem has led to questions about whether physicians are justified in relying on the algorithmic output, with some scholars insisting on the centrality of explainability, while others see no reason to require of AI that which is not required of physicians. We consider that there is merit in both views but find that greater nuance is necessary in order to elucidate the underlying function of explainability in clinical practice and, therefore, its relevance in the context of AI for clinical use. In this paper, we explore explainability by examining what it requires in clinical medicine and draw a distinction between the function of explainability for the current patient versus the future patient. This distinction has implications for what explainability requires in the short and long term. We highlight the role of transparency in explainability, and identify semantic transparency as fundamental to the issue of explainability itself. We argue that, in day-to-day clinical practice, accuracy is sufficient as an “epistemic warrant” for clinical decision-making, and that the most compelling reason for requiring explainability in the sense of scientific or causal explanation is the potential for improving future care by building a more robust model of the world. We identify the goal of clinical decision-making as being to deliver the best possible outcome as often as possible, and find—that accuracy is sufficient justification for intervention for today’s patient, as long as efforts to uncover scientific explanations continue to improve healthcare for future patients.

Reviewing the Potential Role of Artificial Intelligence in Delivering Personalized and Interactive Pain Medicine Education for Chronic Pain Patients

“…AI plays a crucial role in providing guidance and recommendations for treatment options and self-management techniques by analyzing vast amounts of data from various sources, thus generating evidence-based suggestions tailored to individual patients. 25 , 26 For example, an AI-driven platform can analyze a chronic migraine patient’s medical history, socioeconomic and demographic data, symptoms, and genetic markers to recommend specific preventive and abortive medications, complementary therapies, and lifestyle modifications that are associated with optimal treatment outcomes. 27 , 28 AI algorithms can also identify trends and changes in aggregated pain treatment plans and patient therapeutic responses before they would otherwise be discernable to the physician.…”

“…Additionally, transparency and explainability of AI algorithms are vital to establish trust between patients, healthcare providers, and AI systems. 3,25,40,41 Patients should have a clear understanding of how their data are being used, the reasoning behind AI recommendations, and the limitations of the technology. 40 Furthermore, a significant concern as of late is racial bias when training artificial intelligence.…”

A Comprehensive Survey of ArtificialIntelligence in Precision Healthcare:Shedding Light on Interpretability

“…Different end-users require different explanations. For example, AI and ML experts and scientists require explainability at the model or algorithmic level; medical experts require explanations for prediction, inferences or interpretation based on previous cases [62]. Deep learning models are limited in their clinical application due to their black-box nature.…”

Artificial Intelligence in Endoscopic Ultrasonography-Guided Fine-Needle Aspiration/Biopsy (EUS-FNA/B) for Solid Pancreatic Lesions: Opportunities and Challenges

“…One potential solution to this issue is using inherently interpretable models that allow visualization of the regions recognized by AI as being important [84]. Another suggestion is conducting meticulous quality assessments before implementing AI models in the clinic to prevent physicians from relying solely on AI models to evaluate clinical outcomes [85]. This raises intricate regulatory and ethical considerations.…”