What is morally at stake when using algorithms to make medical diagnoses? Expanding the discussion beyond risks and harms

Abstract

In this paper, we examine the qualitative moral impact of machine learning-based clinical decision support systems in the process of medical diagnosis. To date, discussions about machine learning in this context have focused on problems that can be measured and assessed quantitatively, such as by estimating the extent of potential harm or calculating incurred risks. We maintain that such discussions neglect the qualitative moral impact of these technologies. Drawing on the philosophical approaches of technomoral change and technological mediation theory, which explore the interplay between technologies and morality, we present an analysis of concerns related to the adoption of machine learning-aided medical diagnosis. We analyze anticipated moral issues that machine learning systems pose for different stakeholders, such as bias and opacity in the way that models are trained to produce diagnoses, changes to how health care providers, patients, and developers understand their roles and professions, and challenges to existing forms of medical legislation. Albeit preliminary in nature, the insights offered by the technomoral change and the technological mediation approaches expand and enrich the current discussion about machine learning in diagnostic practices, bringing distinct and currently underexplored areas of concern to the forefront. These insights can contribute to a more encompassing and better informed decision-making process when adapting machine learning techniques to medical diagnosis, while acknowledging the interests of multiple stakeholders and the active role that technologies play in generating, perpetuating, and modifying ethical concerns in health care.

Why do errors arise in artificial intelligence diagnostic tools in histopathology and how can we minimize them?

“…There remains uncertainty regarding the use of AI tools in healthcare from a legal, ethical, and regulatory standpoint. If it is inevitable that an AI tool will make a mistake, then a key question is, who is legally responsible―the pathologist, the tool itself, or the implementing trust? 65,66 There is no precedent for this and clarification from the medicolegal community is warranted, encouraging prospective legislative action, rather than reactive (and likely damaging) legal action 67 . There are also numerous ethical challenges with the use of AI.…”

Towards a Taxonomy of AI Risks in the Health Domain

“…At the centre of the ethical discourse are worries and concerns linked to the development, design, and deployment of artificial intelligence systems, which we refer to as ethical risks. This focus has been, however, challenged because it neglects the circumstance that AI systems not only pose challenges to moral goods like values, rights, or duties but also mediate how we conceptualise such moral goods [7].…”

Moral Values in Medical AI: A Scoping Review

“…Domain experts have been observed to hastily rely on AI agent judgments even when instructed to think critically about each judgment [81]. AI may be trusted too readily because of its (at least seeming) objectivity; AI health algorithms replace subjective and fallible human judgments with objective ones based on rigorous data-or so it seems [27]. If su ciently accurate and powerful, AI has the potential to substitute trust for certainty [82].…”

“…and accuracy (or performance, power, etc.). Though not universally seen to really be in con ict, the dilemma between transparency/interpretability and accuracy was far and away the one most discussed by papers in the review [18, 27,30,34,72,78,96,98]. There is some evidence that, despite the seeming importance of transparency, stakeholders actually care more about effectiveness than transparency.…”

“…One domain that is particularly ripe for AI development is medicine. In recent years, many initiatives have arisen to create algorithms that make medical decisions [1,2,2,3,[3][4][5][6][7][8][9][9][10][11][12][13][14][15][16][17][18][19], diagnoses [5,[9][10][11][20][21][22][23][24][25][26][27][28][29][30][31][32][33][34][35][36], and perform other actions historically undertaken by physicians, nurses, hospital administrative staff, and the like. Medical AI has the potential to improve patient wellbeing, health outcomes, and to develop revolutionary new medical technologies.…”