The Challenges of Regulating Artificial Intelligence in Healthcare Comment on "Clinical Decision Support and New Regulatory Frameworks for Medical Devices: Are We Ready for It? - A Viewpoint Paper"

Abstract

Regulation of health technologies must be rigorous, instilling trust among both healthcare providers and patients. This is especially important for the control and supervision of the growing use of artificial intelligence in healthcare. In this commentary on the accompanying piece by Van Laere and colleagues, we set out the scope for applying artificial intelligence in the healthcare sector and outline five key challenges that regulators face in dealing with these modern-day technologies. Addressing these challenges will not be easy. While artificial intelligence applications in healthcare have already made rapid progress and benefitted patients, these applications clearly hold even more potential for future developments. Yet it is vital that the regulatory environment keep up with this fast-evolving space of healthcare in order to anticipate and, to the extent possible, prevent the risks that may arise.

Addressing 6 challenges in generative AI for digital health: A scoping review

“…This allows for more dynamic, emergent behaviors and outcomes, which can provide deeper insights into complex systems, such as oncology trials [ 74 ]. However, medical devices, including AI, need to be approved by regulatory bodies, which requires proving that they are safe and effective [ 75 ], yet many have not been approved. If an AI system is continually changing, it might not remain within the approved parameters.…”

Meeting the challenges of the 21st century: the fundamental importance of trust for transformation

“…Some research has found that patients even prefer responses to medical questions from a chatbot to those from a physician, but clearly context matters [ 50 ]. Yet, once again, caution is needed [ 51 ]. Algorithms developed using data from one population may produce misleading results when applied to another [ 41 ].…”

Artificial intelligence in clinical pharmacology: A case study and scoping review of large language models and bioweapon potential

“…39 • Regulatory complexity: crafting effective regulations requires a profound understanding of AI systems and associated concepts, necessitating nuanced and sophisticated approaches. 40 • Monitoring and enforcement: establishing clear mechanisms to monitor and enforce compliance with the appropriate regulations poses inherent difficulties. 41 • Evolving threats: addressing the ongoing challenge of keeping pace with increasingly complex potential threats to the misuse of AI systems.…”

“…Several noteworthy downsides of AI regulations include: Balancing between innovation and compliance: striking the delicate equilibrium between allowing for AI system improvements and enforcing strict regulations that prevent abusive use 39 Regulatory complexity: crafting effective regulations requires a profound understanding of AI systems and associated concepts, necessitating nuanced and sophisticated approaches 40 Monitoring and enforcement: establishing clear mechanisms to monitor and enforce compliance with the appropriate regulations poses inherent difficulties 41 …”