Ethical and legal considerations influencing human involvement in the implementation of artificial intelligence in a clinical pathway: A multi-stakeholder perspective

Abstract

IntroductionEthical and legal factors will have an important bearing on when and whether automation is appropriate in healthcare. There is a developing literature on the ethics of artificial intelligence (AI) in health, including specific legal or regulatory questions such as whether there is a right to an explanation of AI decision-making. However, there has been limited consideration of the specific ethical and legal factors that influence when, and in what form, human involvement may be required in the implementation of AI in a clinical pathway, and the views of the wide range of stakeholders involved. To address this question, we chose the exemplar of the pathway for the early detection of Barrett's Oesophagus (BE) and oesophageal adenocarcinoma, where Gehrung and colleagues have developed a “semi-automated”, deep-learning system to analyse samples from the CytospongeTM TFF3 test (a minimally invasive alternative to endoscopy), where AI promises to mitigate increasing demands for pathologists' time and input.MethodsWe gathered a multidisciplinary group of stakeholders, including developers, patients, healthcare professionals and regulators, to obtain their perspectives on the ethical and legal issues that may arise using this exemplar.ResultsThe findings are grouped under six general themes: risk and potential harms; impacts on human experts; equity and bias; transparency and oversight; patient information and choice; accountability, moral responsibility and liability for error. Within these themes, a range of subtle and context-specific elements emerged, highlighting the importance of pre-implementation, interdisciplinary discussions and appreciation of pathway specific considerations.DiscussionTo evaluate these findings, we draw on the well-established principles of biomedical ethics identified by Beauchamp and Childress as a lens through which to view these results and their implications for personalised medicine. Our findings are not only relevant to this context but have implications for AI in digital pathology and healthcare more broadly.

Stakeholder perspectives towards diagnostic artificial intelligence: a co-produced qualitative evidence synthesis

“…Across 44 studies, 689 participants were interviewed and 402 participated in focus groups. 31 studies conducted interviews, 16 , 24 , 25 , 31 , 32 , 34 , 36 , 37 , 38 , 40 , 41 , 42 , 43 , 44 , 45 , 48 , 49 , 50 , 51 , 53 , 55 , 57 , 59 , 60 , 61 , 62 , 63 , 64 nine conducted focus groups, 18 , 26 , 27 , 29 , 30 , 33 , 52 , 56 , 58 , 65 five conducted ethnographic studies with interviews, 17 , 46 , 47 , 54 , 66 and two conducted both interviews and focus groups. 35 , 47 The mean number of participants in interview studies was 20 (IQR 15–24), and in focus groups 40(IQR 15–37).…”

Empowering Personalized Pharmacogenomics with Generative AI Solutions

“…[38][39][40] Ethical considerations and regulatory frameworks are additional, well recognized challenges for AI deployment in health care, that need to be addressed. [41][42][43][44][45] Here, we applied methods to reduce the propensity for language biases, inaccuracies, and potential for hallucinations; however, they will nevertheless occur at some frequency.…”

“…Ethical considerations and regulatory frameworks are additional, well recognized challenges for AI deployment in health care, that need to be addressed. [41–45] Here, we applied methods to reduce the propensity for language biases, inaccuracies, and potential for hallucinations; however, they will nevertheless occur at some frequency. When combined with privacy considerations that arise when data are shared in non-restricted environments in order to enable the language models to function, there are clear needs to develop additional approaches to protect patient rights and data security, and maintaining the overall safety and effectiveness of AI applications in healthcare.…”

“I Wonder if my Years of Training and Expertise Will be Devalued by Machines”: Concerns About the Replacement of Medical Professionals by Artificial Intelligence

“…Moreover, this study revealed the evolving impact of AI on patient–provider interactions is an essential aspect of the discussion surrounding AI in healthcare. The concerns about communication challenges with AI reflect a potential source of friction between patients and AI-driven healthcare systems ( Redrup Hill et al, 2023 ). Effective AI–patient communication is vital to ensure patients’ needs are accurately understood and met.…”

“…The complexity of legal implications, especially concerning liability in cases of AI errors, underscores the need for clarity and legal frameworks that address these challenges ( Silva et al, 2022 ). Additionally, Redrup Hill et al (2023) highlights the need for interdisciplinary collaboration in addressing liability challenges. They argue that legal frameworks should be developed with input from experts in both AI and healthcare, ensuring a nuanced understanding of the potential risks and consequences associated with AI errors.…”