A Nationwide Network of Health AI Assurance Laboratories

Abstract

ImportanceGiven the importance of rigorous development and evaluation standards needed of artificial intelligence (AI) models used in health care, nationwide accepted procedures to provide assurance that the use of AI is fair, appropriate, valid, effective, and safe are urgently needed.ObservationsWhile there are several efforts to develop standards and best practices to evaluate AI, there is a gap between having such guidance and the application of such guidance to both existing and new AI models being developed. As of now, there is no publicly available, nationwide mechanism that enables objective evaluation and ongoing assessment of the consequences of using health AI models in clinical care settings.Conclusion and RelevanceThe need to create a public-private partnership to support a nationwide health AI assurance labs network is outlined here. In this network, community best practices could be applied for testing health AI models to produce reports on their performance that can be widely shared for managing the lifecycle of AI models over time and across populations and sites where these models are deployed.

Integrating Artificial Intelligence for Advancing Multiple-Cancer Early Detection via Serum Biomarkers: A Narrative Review

“…On top of a reasonable and explainable MCDE AI model, the next crucial question that physicians would be interested in is the clinical relevance and actionability [ 99 ]. To communicate with patients, the MCED AIs should provide more communicable terms for the predictive results.…”

Back to the Future: Integrating Artificial Intelligence and Automation for Patient Safety

“…These tools must be reliable as well as equitable and widely available to augment (not supplant) high-quality, empathetic care for all. The work must include partnerships among healthcare organizations, technology innovators, researchers, and government agencies like the work that has begun under the Coalition for Healthcare AI (Shah et al, 2024).…”

The potential for artificial intelligence to transform healthcare: perspectives from international health leaders

“…There are efforts to help leaders and health systems develop consensus-based evaluation techniques and infrastructure for AI tools, including HealthAI: The Global Agency for Responsible AI in Health, which aims to build and certify validation mechanisms for nations and regions to adopt; and the Coalition for Health AI (CHAI), which recently announced plans to build a US-wide health AI assurance labs network 7 , 8 . These efforts, if successful, will assist manufacturers and health systems in complying with new laws, rules, and regulations being proposed and released that seek to ensure AI tools are trustworthy, such as the EU AI Act and the 2023 US Executive Order on AI.…”